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Introduction 


1.1  Background 

The  control  and  guidance  of  movement  through  a  structurally  complex  world  presents 
enormous  computational  challenges,  yet  requires  real-time  solutions.  A  hoverfly  making  a 
perfectly  controlled  landing  upon  a  flower  moving  in  the  breeze  demonstrates  these  problems 
are  solved  by  even  simple  biological  systems.  Distinguishing  complex  motions  and  using  this 
information  to  control  locomotion  is  an  important  goal  for  man-made  autonomous  flying 
vehicles,  just  as  it  was  a  strong  selective  pressure  in  the  evolution  of  the  fly’s  visual  system. 
Biomimetic  approaches  to  development  of  electronic  guidance  and  seeker  components,  based 
on  models  of  insect  visual  processing,  are  now  becoming  practical  due  to  advances  in  silicon- 
based  engineering  as  well  as  knowledge  gained  through  extensive  basic  research  on  the 
biological  system  in  recent  years. 

The  research  program  described  here  advanced  this  goal  through  modeling  the  adaptive 
properties  of  the  insect  motion  detection  pathway  and  comparing  the  model’s  response  with 
that  of  the  physiological  system.  This  work  will  lead  to  insights  into  neural  circuits  in  the  insect 
brain  responsible  for  guiding  and  controlling  flight,  and  how  these  cope  with  problems  imposed 
by  the  underlying  mechanism  used  to  detect  and  analyze  optical  flow.  This  ability  is  critical  for 
tasks  such  as  course  control  during  high-speed  pursuit,  and  analysis  of  the  trajectory  of  moving 
targets.  Since  the  insect  visual  system  is  compact  and  utilizes  relatively  sparse  coding  of  the 
visual  input,  it  provides  an  ideal  model  system  for  a  robust,  simple  and  compact  guidance  and 
targeting  system  that  could  be  utilized  by  small,  unmanned  flying  vehicles. 

This  report  documents  an  effort  to  expand  understanding  the  role  of  the  specific 
phenomenon  of  motion  adaptation  in  the  processing  of  wide-field  visual  motion  detection  in 
insects,  particularly  in  the  context  of  the  coding  of  velocity  in  natural  scenes,  and  to  exploit  this 
knowledge  to  develop  biomimetic  computer  models  capable  of  performing  similar  functions. 
The  project  is  a  cross-disciplinary  study  involving  approaches  ranging  from  basic  physiological 
investigation,  through  algorithm  development,  to  implementation  in  software.  Insect  visual 
neurons  display  a  remarkable  ability  to  respond  robustly  to  the  motion  of  both  visual  targets 
and  wide-field  optic  flow,  and  so  this  approach  has  great  potential  for  extension  of 
understanding  from  basic  biology  into  biomimetic  applications  where  similar  tasks  need  to  be 
solved. 

The  focus  of  this  project  was  on  motion  adaptation  that  occurs  in  the  visual  pathway  that 
leads  from  the  retina  to  tangential  neurons  in  the  lobula  plate  (3rd  optic  ganglion)  of  the  fly.  In 
particular,  we  focused  on  the  role  of  this  adaptation  in  promoting  the  phenomenon  of  velocity 
constancy,  which  we  define  to  mean  the  dependence  of  average  tangential  cell  response  on 
velocity  of  optic  flow,  in  combination  with  its  near-invariance  with  respect  to  other  parameters 
of  the  natural  imagery  that  induces  optic  flow  when  the  animal  moves  in  its  environment.  An 
ability  to  model  the  system  and  reproduce  this  behavior  would  provide  a  basis  for  robust, 
simple  artificial  motion  detectors  that  have  a  wide  variety  of  applications  in  both  military  and 
civilian  environments. 

Prior  to  the  commencement  of  this  >3  year  program  of  study,  the  Principal  Investigator  (Dr 
David  C.  O’Carroll)  held  a  position  as  Assistant  Professor  of  Neurobiology  at  the  University  of 
Washington,  Seattle.  Support  from  this  program  enabled  his  re-establishment  of  a  new 
laboratory  in  Adelaide,  Australia,  based  in  the  Department  of  Physiology,  but  with  strong 
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cross-disciplinary  ties  to  the  Department  of  Electrical  and  Electronic  Engineering.  The  first  2 
years  of  this  program  thus  represented  a  major  transition  period  to  this  new  environment  and 
many  technical  hurdles  in  setting  up  new  equipment.  The  success  of  experiments  during  the 
later  period  of  the  program  in  particular  is  due  in  no  small  part  to  the  support  of  this  program. 

In  the  1st  year,  we  performed  pilot  neurobiological  experiments  and  put  forth  a  tentative 
model  for  motion  adaptation,  and  by  conceptual  algorithm  development  in  Matlab, 
demonstrated  the  feasibility  of  a  hiomimetic  implementation.  In  the  2nd  year,  we  consolidated 
the  basic  pilot  neurobiological  studies  in  order  to  begin  probing  the  drivers  for  motion 
adaptation  and  developed  an  adaptive  elementary  motion  detector  (EMD)  model  in  Matlab  and 
tested  and  demonstrated  its  operation  in  one-  and  two-dimensional  arrays  and  a  simulation  of 
yaw.  In  the  3rd  year  we  have  concentrated  modeling  efforts  on  understanding  additional  non¬ 
linear  components  of  motion  processing,  in  an  effort  to  examine  their  roles  in  the  phenomenon 
of  velocity  constancy. 

A  major  bonus  for  the  program  of  research  carried  out  was  a  concurrent  program  funded  as 
a  subcontract  from  Tanner  Research  Inc.,  Pasadena  under  a  Phase  I  and  II  SBIR  contract 
(Contract  F08630-02-C-0013)  from  the  AFRL/MNGN.  The  major  focus  of  the  latter  work  was 
development  of  analog  VLSI  hardware  based  on  the  adaptive  EMD  model,  and  we  made 
substantial  progress  towards  this  goal  over  the  same  period.  An  extension  of  the  performance 
period  in  this  SBIR  project  until  April  this  year  permitted  proposal  and  development  of  a 
transition  path  for  the  biomimetic  models  developed  under  this  project,  into  real-world 
applications.  The  interaction  between  the  two  Pi’s,  Dr  David  O’Carroll  (Adelaide)  and  Dr 
Patrick  Shoemaker  (Tanner)  has  been  a  valuable  and  collegial  collaboration,  of  immense 
benefit  to  the  success  of  the  AOARD  funded  program.  In  particular,  the  SBIR  effort  allowed  us 
to  oversee  the  transfer  of  ideas  from  basic  neuroscience  into  real-world  circuits.  What  is, 
perhaps,  most  valuable  in  this  kind  of  interaction  is  the  feedback  from  the  engineering 
approach  that  suggests  limitations  or  additions  to  the  experimental  approach  in  determining  the 
underlying  algorithms.  The  usefulness  of  such  interactions  during  the  process  of  collecting 
even  basic  neurobiological  data  has  been  of  huge  value  in  ensuring  that  the  best  experimental 
approach  was  adopted. 

1.2  Objectives 

The  project  had  two  central  goals:  to  advance  the  understanding  of  early  elementary  motion 
detectors  (EMDs)  in  the  insect  visual  system  (in  particular,  the  role  of  adaptation)  in  those 
elements,  and  to  develop  biologically  inspired  (and  in  some  cases  biomimetic)  models  for  an 
adaptive  EMD.  Following  are  the  specific  technical  objectives  that  were  formulated  in  support 
of  these  goals: 

The  proposed  work  would  be  carried  out  in  3  overlapping  phases: 

i.  Modeling  the  EMD  using  adaptive  feedback: 

Develop  a  conceptual  model  for  a  set  of  operations  to  implement  a  bio-inspired 
adaptive  mechanism  in  a  software  algorithm  for  an  EMD  based  on  our  recently 
published  data.  Implement  it  in  Matlab  and  test  its  responses  to  animated  natural 
scenes. 

ii.  Analysis  of  physiological  responses  to  broad  band  images. 
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Undertake  further  biological  experiments  to  understand  the  underlying 
mechanisms  of  motion  adaptation.  Test,  modify,  and  elaborate  the  algorithm 
proposed  in  step  I  by  further  neurobiological  experiments. 

iii.  Further  experiments  to  match  model  parameters  to  physiological  responses: 

Elaborate  the  adaptive  EMD  model  into  a  two-dimensional  adaptive  EMD  array 
to  mimic  the  receptive  field  of  a  tangential  HS  neuron  and  test  the  model  using 
stimuli  that  mimic  the  experiments  executed  under  (ii).  Adjust  model  parameters 
to  explore  properties  of  the  adaptive  model. 
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1.3  Executive  Summary:  Neurobiology 

1.3.1  Aims  and  Approach 

Our  earlier  pilot  neurobiological  recordings  (carried  out  at  the  University  of  Cambridge, 
UK,  in  1998)  established  a  group  of  tangential  neurons  in  the  lobula  complex  of  the  hoverflies 
Eristalis  tenax  and  Volucella  pelluscens  as  an  ideal  model  for  investigating  the  coding  of 
velocity  in  insect  motion  detection  and  the  role  that  adaptation  plays  in  this  process.  Our  pilot 
data  was  strongly  suggestive  of  an  unusual  degree  of  ‘velocity  constancy’  -  i.e.  invariance  with 
respect  to  both  contrast  and  specific  image  statistics  in  the  neuron  response  as  a  function  of 
velocity.  Such  robust  responses  would  be  a  highly  desirable  characteristic  of  biomimetic 
motion  detectors  in  artificial  vision  applications.  Because  such  robustness  is  not  characteristic 
of  a  system  based  on  classical  biological  models  for  motion  detection,  we  considered  this  result 
to  be  of  vital  importance  for  further  study.  However,  our  main  experimental  approach  (in-vivo 
intracellular  recording  from  single  neurons  in  a  flight  simulator)  is  technically  demanding  and 
our  pilot  data  reflected  the  experimental  recording  error  (mainly  noise)  that  characterizes  this 
approach.  The  stimuli  used  in  those  experiments  were  also  relatively  primitive,  being  based  on 
artificially  generated  random  texture  patterns  rather  than  true  ‘natural’  images  of  the  kind  that 
would  be  encountered  by  either  natural  or  artificial  seeing  systems  in  real-world  applications.  A 
major  aim  of  our  Neurobiological  experiments  was  therefore  to  verify  these  earlier 
observations  through  repeated  averaging  of  responses  a  larger  set  of  images  obtained  from 
natural  scenes,  and  then  to  begin  the  process  of  investigating  the  underlying  mechanisms  and 
drivers  for  adaptation  through  further  experiments. 

Methods:  For  our  main  experimental  effort  we  selected  a  subset  of  tangential  cells,  the  HS 
neurons,  which  are  a  yaw-selective  element.  These  HS  neurons  were  selected  because  they 
have  receptive  fields  that  are  believed  to  be  tuned  to  the  yaw  component  of  optical  flow 
(although  some  recent  work  questions  this  interpretation).  Using  a  digital  camera,  we  are  able 
to  digitize  panoramic  scenes  and  stitch  these  into  a  continuous  cylindrical  projection  of  the 
surrounding  world  that  can  be  animated  in  prolonged  sequences  that  are  reasonably  appropriate 
to  the  world  that  would  be  viewed  by  such  neurons  under  natural  conditions.  A  major  effort  by 
Andrew  Straw  (a  PhD  student  supported  by  a  Howard  Hughes  Medical  Institute  Predoctoral 
Fellowship)  during  the  earlier  part  of  this  period  established  a  new  set  of  experimental  stimuli 
via  a  suite  of  open-source  function  calls  built  on  OpenGL  to  an  animation  engine  optimized  for 
visual  stimulus  presentation.  Andrew  adopted  the  name  “VisionEgg”  for  this  project  and 
development  of  refined  versions  of  these  stimuli  continues  to  date.  Although  the  VisionEgg 
development  had  commenced  prior  to  the  present  project,  it  became  a  core  component  of  it  and 
refinement  of  this  software  continued  throughout  the  project.  The  software  itself  incorporated  a 
number  of  open-source  Python  libraries  (in  particular  a  module,  PyGame,  original  designed  as 
an  API  for  computer  games  applications).  Andrew’s  contribution  to  this  effort  was  a  valuable 
one,  and  he  has  released  his  project  under  the  terms  of  the  open-source  LGPL  for  free 
download  (www.visionegg.org).  It  has  subsequently  been  adopted  by  a  number  of  laboratories 
for  experimental  stimulus  development  and  applications. 

In  the  experiments  forming  the  basis  of  this  project,  VisionEgg  animated  panoramas  were 
presented  on  a  high-speed  CRT  monitor  mounted  within  the  receptive  field  of  the  neuron  and 
subtending  approximately  1 00  degree  field  of  regard,  at  a  frame  rate  of  200Hz.  The  real-time 
rendering  of  the  OpenGL  engine  permits  interpolation  of  the  panoramic  images  so  that  even 
very  low  speeds  of  motion  can  be  simulated  robustly.  We  implemented  complex  spatial  and 
temporal  anti-aliasing  methods  to  limit  any  display  artifacts.  Additional  software  allowed  us  to 
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synchronize  stimulus  presentation  with  the  digitization  of  the  membrane  potential  of  HS 
neurons  after  pre-amplification. 

Experimental  design:  We  used  several  different  stimulus  paradigms  based  on  the  same 
cylindrical  projections  of  animated  textures.  Drifting  sinusoidal  grating  patterns  that  were 
ramped  in  contrast  over  time  permitted  estimation  of  the  local  contrast  gain  of  the  neuron 
within  its  receptive  field  (‘local  sensitivity’)  as  well  as  basic  spatial  and  temporal  tuning 
properties  of  each  neuron  -  a  technique  that  we  had  used  to  considerable  effect  in  our  earlier 
studies  (e.g.  Harris,  O’Carroll  &  Laughlin  2000).  Velocity  constancy  was  established  through  a 
sequence  of  interleaved  test  and  adapting  ‘bursts’  of  simulated  yaw  for  natural  panoramic 
images,  permitting  quantitative  evaluation  of  the  velocity  tuning  of  the  neuron  under  strongly 
adapted  conditions.  Further  experiments  aimed  at  testing  the  underlying  drivers  of  motion 
adaptation  using  variants  of  a  ‘probe-adapt-probe’  protocol  that  allowed  us  to  evaluate  the 
contrast  gain  of  the  neuron  before  and  after  application  of  a  specific  adapting  stimulus. 

1.3.2  History 

Much  initial  work  on  this  project  also  formed  a  major  part  of  the  PhD  thesis  work  of 
Andrew  Straw  (with  salary  support  from  a  concurrent  Howard  Hughes  Medical  Institute 
Fellowship).  The  main  part  of  his  thesis  work  relevant  to  this  effort  focused  on  documenting 
velocity  constancy  in  the  yaw  simulator  experiments  and  primarily  in  a  motion-adapted  state 
(using  the  interleaved  stimulus  referred  to  above).  As  his  tenure  reached  completion,  some 
experiments  were  incomplete  and  were  fleshed  out  through  further  recordings  by  Tamath 
Rainsford  (with  partial  salary  support  from  both  SBIR  and  AOARD  funded  projects)  allowing 
us  to  verify  the  conditions  under  which  velocity  constancy  was  observed.  Tamath  also 
completed  experiments  aimed  at  evaluating  velocity  constancy  in  ‘unadapted’  neurons.  Once 
this  was  established  Tamath  moved  on  to  experiments  using  the  probe-adapt-probe  protocol  to 
try  and  quantify  gain  changes  following  adaptation  with  different  drivers  (specifically  the 
velocity  or  contrast  of  the  adapting  stimulus).  Although  we  had  initially  hoped  to  develop  a 
simplified  method  to  evaluate  gain  change  and  thus  to  explore  the  adapting  stimulus  parameter 
space  extensively,  experimental  recording  noise  plagued  our  early  efforts  on  this  task.  Some 
effort  was  put  into  refining  this  latter  stimulus  to  optimize  our  ability  to  evaluate  contrast  gain 
change  following  adaptation  in  a  minimum  of  neuron  recording  time  (which  is  always  the 
limitation  on  the  amount  of  data  that  can  be  obtained  in  biological  experiments)  by  developing 
analysis  techniques  based  on  a  Weibull  function  fit.  While  successful,  this  approach  was  more 
time  consuming  than  we  had  hoped,  and  so  complete  exploration  of  the  dependence  of  gain 
changes  on  contrast  and  the  speed  of  the  adapting  pattern  are  ongoing.  In  the  last  6  months  of 
the  experimental  program,  have  started  looking  on  more  detail  at  the  ‘waterfall’  effect  and  the 
speed  of  motion  adaptation.  Briefly,  most  of  our  earlier  experiments  and  indeed  our  modeling 
efforts  have  focused  on  contrast  gain  control  as  the  primary  component  of  motion  adaptation. 
But  since  our  earlier  work  on  this  system  had  suggested  a  prominent  and  apparently  separable 
component  of  adaptation  -  the  ‘waterfall  effect’  might  also  play  a  role  in  velocity  coding,  we 
commenced  analysis  of  its  dependence  on  parameters  of  the  stimulus  and  its  possible  effects  on 
effective  gain. 

1.3.3  Results  and  Conclusions 

1.  Velocity  constancy  in  motion-adapted  neurons.  Good  velocity  constancy  was 
observed  in  adapted  responses  to  sustained  yaw  stimuli.  Velocity  tuning  shows  a 
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consistent  response  up  to  velocities  of  ca.  200  degrees/second  irrespective  of  the 
image  selective,  providing  it  is  statistically  natural.  Curves  from  a  variety  of  such 
images,  including  two  derived  from  interior  of  man-made  buildings,  nearly  overlie 
one  another.  Yet  when  narrow-band  (sinusoidal)  stimuli  with  different  spatial 
frequencies  are  presented,  they  confound  this  response  in  a  manner  consistent  with 
the  classical  Hassenstein/Reichardt  correlation  model. 

2.  Velocity  constancy  in  unadapted  neurons.  Initially  applied  as  a  control  stimulus, 
our  attempts  to  measure  responses  of  unadapted  neurons  (i.e.  those  that  had  viewed  a 
blank  display  for  several  seconds  prior  to  stimulation)  also  showed  a  surprising 
degree  of  velocity  constancy.  This  finding,  substantiated  approximately  half-way 
through  this  project,  and  yet  contradicted  by  any  ‘simple’  models  based  on  the 
Hassenstein/Reichardt  type  EMD,  raised  interesting  questions  about  the  role  of 
additional  non-linear  processing  in  velocity  constancy.  As  a  caveat,  our  analysis 
technique  requires  us  to  discard  transient  responses  (over  at  least  the  time  constant  of 
the  underlying  neural  delay  mechanisms  inherent  to  any  motion  detection  algorithm) 
and  thus  raises  questions  about  the  possible  role  of  fast  temporal  adaptation. 
Experiments  designed  to  test  these  ideas  were  commenced  at  the  end  of  this  project 
(see  6  below). 

3 .  Effect  of  artificially-scaled  contrast  on  velocity  constancy:  Our  recordings 
demonstrated  velocity  constancy  with  natural  images  with  a  large  variety  of  global 
contrasts,  by  any  of  several  measures  that  we  applied.  One  conclusive,  yet  enigmatic 
finding  is  that  artificially  reducing  contrast  of  any  of  these  images  results  in  a  break¬ 
down  in  velocity  constancy.  Re-scaled  images  give  consistently  weaker  responses. 
Interestingly,  truly  natural  scenes  still  give  similar  curves  to  one  another,  while  the 
artificial  (interior)  scenes  generate  less  attenuated  responses. 

4.  Contrast  &  Velocity  dependency  of  adaptation:  We  show  that  the  degree  of  gain 
reduction  during  motion  adaptation  is  dependent  on  contrast  of  the  adapting  pattern. 
The  shape  of  this  dependence  is  supra-linear  in  the  same  manner  as  that  of  the  EMD 
response  itself.  Interestingly,  adaptation  is  also  clearly  dependent  on  the  velocity  of 
motion,  but  not  in  a  manner  directly  predicted  by  the  shape  of  velocity  tuning  in  the 
unadapted  state.  High  speeds  that  excite  little  net  response  from  the  HS  neuron  cause 
strong  adaptation,  suggesting  that  adaptation  is  not  directly  driven  by  the  output  of 
local  motion  detectors.  A  caveat  to  this  conclusion  is  that  if  (as  suggested  above)  fast 
components  of  adaptation  play  a  role  in  shaping  responses  that  we  measure,  our 
attempts  to  evaluate  responses  of  the  unadapted  neuron  may  themselves  be 
confounded  by  the  effects  of  adaptation.  Nevertheless,  some  pilot  experiments  using 
ramps  of  contrast  to  produce  a  ‘soft’  onset  to  stimulation  and  sinusoidal  grating 
stimulus  patterns  also  suggest  that  adaptation  is  more  prominent  at  higher  temporal 
frequencies  than  are  optimal  drivers  for  the  primary  motion  response.  These  data  are 
consistent  with  a  feed-forward  gain  control  mechanism  that  is,  nevertheless, 
regulated  by  feedback  from  motion  detector  outputs 

5.  Role  of  the  Waterfall  effect:  Our  experiments  show  that  the  waterfall  effect  is  much 
more  likely  than  the  gain-reduction  to  be  dependent  on  the  properties  of  the  HS 
neuron  response  itself.  It  appears  to  contribute  a  long-lasting,  DC  potential  to  the 
tangential  cell  output,  and  antagonistic  to  the  global  activity  of  the  neuron  during 
stimulation.  Unlike  gain  reduction,  both  velocity  tuning  and  contrast  dependence  of 
this  component  match  those  of  the  unadapted  HS  neuron.  We  thus  conclude  that  the 
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waterfall  component  is  intrinsic  to  the  motion  computation  process  itself,  is  a 
separate  mechanism  to  the  gain  control  and  works  additively  with  respect  to  the 
effects  of  the  latter.  Because  of  this,  this  component  can  exert  a  global  influence  on 
the  properties  of  a  tangential  cell,  while  the  gain  reduction  is  strictly  local 
phenomenon. 

6.  Speed  of  adaptation:  Our  initial  experiments  show  that  adaptation  may  be  fast 
enough  to  explain  the  velocity  constancy  observed  in  apparently  ‘unadapted’ 
neurons.  Our  pilot  studies  are  incomplete  and  imperfectly  designed  experiments  at 
the  time  of  writing,  and  we  will  seek  funds  to  continue  this  work  in  future  projects. 


1.4  Executive  Summary:  Simulation  and  Modeling 

1.4.1  Aims  and  Approach 

Our  simulation  and  modeling  efforts  aimed  to  achieve  two  things:  to  better  explain  the 
features  observed  experimentally  in  the  biological  visual  motion  processing  pathway, 
especially  motion  adaptation,  and  to  investigate  in  detail  their  roles  in  producing  useful 
information  about  optic  flow  /  visual  motion,  particularly  with  regard  to  producing  a  model  for 
velocity  constancy.  Models  were  built  around  a  model  for  an  elaborated  elementary  motion 
detector  (EMD)  as  we  had  used  in  an  earlier  study  of  based  on  the  Hassenstein-Reichardt 
model  (Dror,  O’ Carroll  &  Laughlin  2000),  with  the  addition  of  motion  adaptation  (as  will  be 
detailed  later).  An  important  feature  of  this  elaborated  EMD  model  that  is  missing  from  many 
earlier  modeling  studies  is  that  it  incorporates  models  for  non-linear  coding  by  “early”  visual 
processing  (i.e.  by  photoreceptors  and  first  order  interneurons),  and  also  for  non-linearity  in 
spatial  integration  of  local  motion  signals  by  tangential  cells  which  collate  EMD  outputs.  This 
approach  allowed  us  to  compare  the  output  of  simulations  of  the  entire  processing  chain  which 
that  of  the  biological  system  that  was  subjected  to  experimental  observation  (in  the  form  of 
recordings  from  tangential  neurons).  As  our  earlier  work  (Harris  et  al  2000)  had  identified  two 
distinct  components  of  motion  adaptation,  our  comparison  focused  primarily  on  both  of  these:  a 
motion-dependent  gain  modulation  and  a  “DC  offset”  component  termed  the  ‘waterfall  effect’ 
(persistent  hyperpolarization).  The  latter  was  modeled  primarily  in  the  later  phases  of  this 
project. 

In  our  concurrent  SBIR  funded  modeling  and  electronic  simulation  work,  we  developed 
models  for  gain  modulation  primarily  based  on  an  easily-implemented  feedforward  contrast 
gain  control.  Our  modeling  for  this  effort  tested  an  alternative  strategy,  for  which  we  had  some 
evidence  in  experimental  data,  based  on  feedback  of  local  pooled  motion  signals.  In  addition  to 
modeling  a  feedback  gain  control  mechanism  driven  by  a  bi-directional  (rectified)  sum  of  local 
motion  outputs,  we  explored  an  initial  (and  somewhat  crude)  implementation  of  the  waterfall 
effect  as  a  separate  offset  introduced  into  local  EMDs  based  on  their  individual  (unrectified) 
outputs.  Experimental  evidence  to  date  is  inconclusive  as  to  whether  the  waterfall  component  is 
a  local  or  global  property  within  HS  neuron  collation  of  local  EMDs,  but  even  a  local 
implementation  of  this  kind  produces  a  global  influence  on  the  collated  EMD  output,  and  thus 
is  consistent  with  all  experimental  data. 

The  environment  used  for  all  modeling  in  this  effort  was  MATLAB,  although  we  carried 
out  some  stages  (spatial  pre-filtering)  using  Adobe  Photoshop. 

Simulations  were  generally  performed  with  animated  natural  imagery,  in  order  to  maintain 
the  spatial  statistics  seen  by  the  biological  system,  and  with  relatively  large  arrays  of  pixels  and 
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EMDs,  so  that  receptive  fields  of  reasonable  size  could  be  constructed  for  wide-field 
integrating  neurons  (tangential  cells).  The  modeling  approach  used  the  same  set  of  panoramic 
images  as  used  in  the  neurobiological  experiments. 

1.4.2  History 

Simulation  tools  were  developed  and  simulations  run  throughput  the  project.  Basic 
implementation  of  a  2-d  EMD  array  was  completed  during  first  2  years  of  the  project  and 
elaboration  of  this  model  to  incorporate  the  waterfall  component  was  complete  by  the  end  of 
the  3  rd  year. 

1.4.3  Results  and  Conclusions 

The  feedback  modeling  approach  reproduces  a  number  of  interesting  aspects  of  the 
biological  system  and  certainly  provides  a  much  robust  estimate  of  pattern  velocity  than  other 
variants  of  the  correlational  EMD.  However,  neither  the  model  developed  here,  nor  the 
concurrent  modeling  effort  in  the  SBIR  funded  project  fully  explain  the  degree  of  velocity 
constancy  seen  in  the  biological  system,  suggesting  that  further  refinement  is  necessary. 
Contrast  gain  control  greatly  reduces  the  dependence  of  response  on  image  contrast,  but  still 
not  to  the  degree  seen  in  biological  neurons,  and  it  does  not  explain  the  fact  that  adaptation  is 
more  pronounced  in  response  to  motion  signals  versus  flicker  stimuli.  As  implemented,  the 
feedback  model  is  also  relatively  sluggish  in  achieving  velocity  constancy,  whereas  our  later 
biological  experiments  showed  that  this  property  is  evident  in  even  the  first  few  hundred 
milliseconds  of  the  response  of  the  neuron  to  the  onset  of  motion.  Furthermore,  the  feedback 
approach  does  not  approximate  velocity  constancy  unless  assumptions  (such  as  high  feedback 
gain)  are  made  which  tend  to  distort  the  velocity  dependence  of  response  such  that  it  no  longer 
closely  resembles  that  of  the  neurobiological  system.  That  said,  the  model  incorporates 
substantial  dynamic  non-linearity  and  is  computationally  expensive  in  our  present  simulation 
environment,  making  it  difficult  to  fully  explore  the  large  range  of  potential  variables  that 
influence  the  match  between  model  and  experimental  data.  In  addition,  it  does  not  account  for 
fact  that  very  high-speed  motion  induces  profound  adaptation  even  though  motion  signals  (i.e., 
tangential  cell  responses)  are  weak  under  these  conditions.  It  does,  however,  fully  explain  the 
difference  between  motion-  and  flicker-induced  adaptations,  a  feature  that  remains 
inadequately  captured  by  our  concurrent  modeling  efforts  with  Tanner  research  (and  funded  by 
the  SBIR). 

There  are  two  possible  conclusions  of  the  modeling  effort  that  encourages  further 
elaboration  of  the  models  proposed  and  tested  in  this  effort.  The  first,  and  most  obvious,  is  that 
motion  adaptation  is  not  a  single  process  but  rather  has  several  separable  components  operating 
in  parallel.  A  combination  of  both  feedforward  and  feedback  models,  for  example,  would  be 
consistent  with  all  of  the  biological  data  obtained  to  date.  A  caveat  to  this  conclusion  is  that 
such  a  complex  elaboration  might  not  represent  the  simplest  solution  to  the  task  and  is  not  in 
itself  supported  by  any  direct  evidence  from  experiments,  so  we  should  thus  be  cautious  in 
claiming  such  a  hybrid  model  as  biomimetic.  The  second  is  experimental  evidence  for  the 
separability  of  the  waterfall  component  of  adaptation.  Inclusion  of  this  effect  in  our  modeling 
of  the  feedback  mechanism  certainly  provides  excellent  fits  to  the  results  of  probe-adapt-probe 
experiments.  More  importantly,  this  also  improves  velocity  constancy  of  model  outputs.  A 
similar  caveat  applies  to  this  conclusion,  since  at  this  stage  the  form  of  feedback  used  to 
implement  the  waterfall  effect  in  our  models  is  not  based  on  any  direct  experimental  evidence, 
and  thus  should  be  considered  bio-inspired  rather  than  biomimetic:  Much  further 
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neurobiological  experimentation  is  required  to  clarify  the  underlying  mechanisms  of  the 
waterfall  component. 
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2  Project  Execution  and  Results 

Details  of  project  execution  and  results  are  given  in  this  section  for  each  of  the  major 
technical  areas  of  Neurobiology  &  Simulation  /  Modeling.  They  are  arranged  in  approximate 
chronological  order. 

2.1  Neurobiology 

2.1.1  Year  1 

Neurobiological  efforts  during  the  first  year  of  activity  were  dominated  by  the  demands  of 
constructing  the  new  laboratory  in  Adelaide.  By  the  time  of  a  site  visit  by  AOARD  program 
manager  (October  10,  2001)  PhD  student  Andrew  had  joined  the  PI  (David  O’Carroll)  in 
Adelaide,  we  had  been  allocated  temporary  lab  space  in  the  Department  of  Physiology  and 
were  beginning  to  order  equipment  for  the  project  (aided  by  AUD$70,000  in  setup  funds  from 
the  University  of  Adelaide).  Construction  of  the  main  laboratory  space  for  the  neurobiological 
experiments  did  not  commence  until  early  2002. 

Substantial  effort  by  Andrew  Straw  on  a  development  setup  allowed  us  to  test  the  emergent 
VisionEgg  software  platform  however.  By  mid  2002  we  were  able  to  move  into  the  new 
laboratory  and  had  constructed  an  initial  experimental  setup.  We  also  collected  a  set  of 
panoramic  image  sets  to  be  used  in  subsequent  experiments  and  stitched  these  into  single 
panoramic  textures  that  could  be  animated  in  the  VisionEgg  system.  By  the  time  the 
experimental  setup  and  software  were  functional,  it  was,  however,  mid-winter  and  thus  no 
animals  were  available  for  experiments. 


2.1.2  Year  2 

With  the  arrival  of  Spring  (October  2002),  our  efforts  turned  to  location  of  a  suitable  source 
of  experimental  animals  for  initial  testing.  Volucella  pelluscens  is  not  available  in  Adelaide,  but 
the  second  species  studied  in  the  earlier  experiments,  Eristalis  tenax  was  located  (with  some 
difficulty)  in  the  hill  country  surrounding  the  city.  We  therefore  concentrated  our  efforts  on 
locating  suitable  field  sites  for  obtaining  experimental  animals  of  this  species.  We  eventually 
located  a  consistent  source  of  E.  tenax  in  the  Mt  Lofty  Botanic  gardens  and  applied  for  a 
collecting  permit  from  the  manager.  This  was  granted  and  we  are  indebted  to  the  staff  of  the 
Botanic  gardens  who  have  been  very  helpful  in  allowing  us  access  to  their  gardens  for 
collecting  purposes  throughout  this  project. 

With  a  source  of  flies  secured,  we  ran  a  number  of  initial  experiments  in  which  adaptation 
was  studied  by  observation  of  HS  and  VS  cell  responses  to  the  test/adapt/test  stimulation 
protocol  used  in  our  earlier  work  (Harris  et  al  2000),  but  utilizing  the  natural  images  for  the 
first  time.  This  protocol  was,  however,  extended  in  an  effort  to  provide  data  that  will  help  to 
unconfound  the  separate  effects  of  EMD  gain  adaptation  and  lobula  plate  cell  hyperpolarization 
(the  “waterfall  effect”),  and  to  test  hypothesis  regarding  the  adaptive  mechanism.  The  modified 
protocol  involved  application  of  adapting  stimuli  in  both  the  preferred  and  antipreferred 
directions,  rather  than  the  preferred  direction  alone.  Antipreferred-direction  stimuli  have  been 
shown  to  elicit  essentially  the  same  gain  reduction  as  preferred-direction  stimuli,  but  without 
inducing  the  waterfall  effect.  This  approach  was  taken  in  an  effort  to  determine,  for  example, 
whether  depression  of  response  seen  during  adapt  phases  during  the  test-adapt-test  protocol 
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results  from  the  waterfall  effect  or  of  gain  reduction,  and  the  degree  to  which  the  waterfall 
effect  itself  might  contribute  to  velocity  constancy  (given  that  it  is  currently  lacking  from  our 
models). 

2. 1.2.1  Confounding  effects  of  pattern  noise 

Unfortunately,  the  experiments  during  this  period  were  not  conclusive,  due  to  a  second 
confounding  effect.  HS  cell  responses  in  such  experiments  display  a  significant  amount  of 
“pattern  noise,”  or  dependence  on  the  spatial  structure  of  the  stimulus  that  happens  to  be 
passing  by  the  relevant  portion  of  visual  space  during  the  relatively  short  test  phases.  This  is  an 
intrinsically  interesting  observation,  confirming  our  earlier  modeling  predictions  (Dror  et  al 
2000)  that  pattern  noise  remains  one  of  largest  potential  problems  for  ‘real-world’  applications 
of  an  elaborated  EMD  array,  despite  the  statistical  predictability  of  natural  scenes  in  a  global 
sense.  The  basic  phenomenon  of  pattern  noise  is  exacerbated  by  our  experimental  stimulus. 
Preliminary  modeling  efforts  suggested  that  the  large  HS  cell  receptive  field  is  specifically 
‘designed’  to  reduce  this  effect  in  nature,  by  elongation  along  the  axis  for  the  preferred 
direction  of  optic  flow  that  the  neuron  is  designed  to  detect.  This  elongated  receptive  field  is 
further  extended  at  the  output  of  the  tangential  cells  in  the  steering  apparatus  of  the  fly  thorax, 
where  summation  of  signals  from  the  two  sides  of  the  body  serve  to  extend  the  effective 
receptive  field  even  further.  In  experiments,  however,  stimuli  were  presented  on  a  computer 
display  that  stimulates  only  a  subset  of  that  receptive  field  (around  90  degrees  in  width).  When 
the  stimulus  was  run  “backward”  during  the  adapt  phases  in  the  antipreferred  direction,  this 
structure  was  different,  and  the  resulting  difference  in  pattern  noise  masked  the  effect  that  we 
were  looking  for. 

We  developed  an  approach  to  address  the  problem  by  reflecting  the  images  horizontally 
during  each  adapting  stimulus  phase  so  that  the  spatiotemporal  structure  of  the  stimuli  could  be 
made  identical  for  stimuli  in  either  direction.  We  also  ‘split’  the  panoramic  images  horizontally 
into  4  segments  with  90  degrees  ‘phase’  difference  between  them.  This  reduced  the  effect  of 
vertical  phase-aligned  features  that  are  prominent  in  natural  images,  whilst  still  stimulating  the 
neurons  with  images  that  had  basically  ‘natural’  image  statistics.  Note  that  the  (1/f n)  image 
statistics  that  are  consistent  for  all  natural  scenes  ignores  such  phase  correlations  within  stimuli 
and  that  such  correlations  would  have  no  effect  on  the  behavior  of  local  adaptation  (adaptation 
had  been  demonstrated  to  be  local  in  very  early  work  by  Maddess  &  Laughlin,  1986)  since  they 
are  effects  that  extend  beyond  the  receptive  field  of  individual  EMDs. 

Despite  our  efforts,  the  presence  of  “pattern  noise”  remained  a  significant  confounding 
influence  on  the  results  of  the  initial  experiments.  We  began  working  on  an  elaboration  of  the 
test  procedure,  involving  registration  of  uninverted  and  mirrored  imagery  during  various 
portions  of  the  experiments,  which  we  expected  to  allow  better  rejection  of  “pattern  noise” 
when  computing  the  direction-selective  portion  of  adaptation  (the  “waterfall”  effect). 
Computation  of  the  non-directional  effect  (i.e.,  gain  reduction)  would  still  require  signal 
averaging  over  various  phases  of  image  presentation.  Unfortunately,  the  condition  of 
uniformity  of  EMD  properties  (i.e.,  connection  strength  onto  the  HS  cell)  across  the  HS 
receptive  field  is  not  well-approximated  in  the  biological  neuron:  instead  the  receptive  field  is 
strongly  asymmetric,  with  maximal  sensitivity  close  to  the  frontal  part  of  the  receptive  field. 
Although  data  loading  procedures  for  the  ‘Vision  Egg’  were  developed  for  this  modification,  it 
was  not  ultimately  used  extensively  in  future  experiments.  Instead,  we  adopted  a  standard 
procedure  that  was  much  more  laborious:  By  presenting  all  stimuli  at  a  variety  of  initial  phases 
and  averaging  across  many  repetitions,  the  effects  of  pattern  noise  are  mitigated.  Unfortunately, 
this  largely  insoluble  problem  leads  to  a  large  increase  in  the  total  number  of  experiments 
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required  and,  ultimately,  to  a  reduction  in  the  amount  of  useful  data  that  can  be  obtained  in 
even  the  best  intracellular  recordings  (with  a  duration  up  to  several  hours). 

2. 1.2. 2  Elaboration  of  VisionEgg  stimuli 

During  this  period  we  also  completed  new  stimulus  generation  techniques  relying  on  the 
VisionEgg.  As  before,  the  stimuli  were  derived  from  panoramic  images  moved  in  the  yaw 
direction,  but  using  the  OpenGL  tools  and  hardware  accelerated  functions  available  from  the 
Graphics  card  in  the  stimulus  PC,  we  began  to  predistort  them  for  perspective  (rather  than 
projecting  them  flat  on  the  CRT)  so  that  they  corresponded  more  accurately  with  pure  yaw 
from  the  position  of  the  test  animal.  In  this  configuration  the  monitor  effectively  models  a 
viewport  onto  a  virtual  cylinder.  Additional  advantages  include  the  ability  to  do  real-time 
interpolation  of  the  imagery  on  a  frame-by-frame  basis  at  the  high  monitor  refresh  rate  (200Hz) 
required  to  overcome  flicker  sensitivity  in  the  fast  fly  visual  system.  This  greatly  improved 
animation  quality  for  very  low  speed  animation,  where  the  image  needs  to  move  a  fraction  of  a 
pixel  between  successive  video  frames:  Our  previous  display  system  had  been  limited  to 
movement  of  the  image  by  a  minimum  increment  of  2  pixels,  introducing  ‘jumping’  artifacts 
into  low  speed  animations.  Other  new  features  included  the  use  of  true  color  (24  bit)  imagery. 

Initial  application  of  these  stimuli  to  intracellular  recordings  from  3  HS  neurons  in  Eristalis 
did  not  appear  to  support  velocity  constancy  as  strongly  as  earlier  data,  although  it  was  clear 
that  adaptation  resulted  in  significant  compression  of  the  output  range  relative  to  the  quadratic 
variation  seen  in  the  idealized  Reichardt  detector.  With  the  approach  of  winter  and  associated 
difficulty  obtaining  Eristalis,  emphasis  was  therefore  on  experiments  to  complete  analysis  of 
adaptation  to  natural  images  in  the  'monotonic'  velocity  tuning  experiments  as  illustrated  in 
figure  1  below: 


Figure  1:  HSNE  response  to  natural  images  animated  to  simulate  yaw  motion  in 
monotonic  series. 
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This  stimulus  (see  lower  trace)  consisted  of  an  initial  3  second  period  of  motion  of  an 
adapting  pattern  followed  by  a  series  of  interleaved  velocities,  presenting  in  ascending  and  then 
descending  order,  interleaved  at  200ms  intervals  with  a  ‘burst’  of  the  same  adapting  speed  as  in 
the  initial  period.  The  upper  trace  in  this  figure  shows  the  recorded  membrane  potential  of  an 
HS  neuron  during  a  single  trial  presentation  of  this  stimulus,  while  the  middle  trace  shows  the 
average  of  several  trials,  presented  at  different  initial  positions  of  the  panoramic  image  in  order 
to  mitigate  the  effects  of  pattern  noise,  as  described  earlier.  Even  in  the  raw  data,  the  velocity 
sensitivity  of  the  HS  neuron  is  evident,  with  a  pronounced  optimum  visible  in  both  the 
increasing  and  decreasing  test  series. 

One  interesting  observation  came  from  a  rare  recording  from  the  'northernmost'  of  the  set  of 
HS  neurons,  HSN.  We  recorded  from  this  neuron  infrequently  because  of  the  typical 
orientation  of  our  preparation  with  respect  to  the  axon  position  of  this  cell.  Based  on 
preliminary  analysis  on-line,  we  noted  that  there  were  discrepancies  between  velocity  tuning 
obtained  with  an  increasing  series  of  test  speeds,  compared  with  that  on  the  second  half  of  each 
series  presented,  where  test  speeds  are  initially  high  and  gradually  decrease  (over  a  12  second 
period).  We  previously  used  the  strict  'symmetry'  in  such  curves  as  evidence  that  our  adapting 
pattern  (interleaved  with  the  tests)  maintains  a  constant  adaptation  state  throughout  the 
experiment,  and  in  typical  HS  recordings  from  the  more  frequently  encountered  HSNE  and 
HSE  neurons,  this  was  the  case.  Our  results  for  HSN  suggested  that  the  time  constant  for  the 
adaptive  process  in  this  cell  may  be  much  longer.  We  tested  this  by  increasing  the  initial 
adaptation  period  from  3  seconds  to  8  seconds,  and  this  then  produced  near  perfect  symmetry 
in  the  responses,  confirming  that  it  takes  approximately  8  seconds  for  responses  to  reach  a 
'steady  state'  compared  with  only  2-3  seconds  in  HSNE  and  HSE.  The  functional  significance 
of  this  was  not  clear,  given  that  modeling  efforts  at  the  time  suggested  that  that  the  time  taken 
for  response  to  reach  steady  state  is  not  necessarily  related  to  time  constant  of  the  low-pass 
filter  in  the  adaptive  element,  at  least  not  where  there  is  significant  feedback  in  the  adaptation 
process. 

The  purpose  of  the  monotonic  presentation  is  to  evaluate  the  effect  of  the  test  stimuli  (i.e. 
the  variable  velocities)  on  the  adaptation  state  of  the  neuron.  The  assumption  is  that  powerful 
motion  adaptation  by  the  adapting  stimulus  ‘clamps’  the  adaptation  state  to  a  fixed  level.  This 
can  be  evaluated  by  extracting  the  response  to  adapting  periods  during  each  interleaved  burst 
and  plotting  them  against  the  test  velocity  that  precedes  them,  as  in  figure  2  below.  If 
adaptation  state  is  kept  constant,  we  should  see  a  flat  line  for  the  adapting  test  patterns. 
Initially,  we  considered  using  a  random,  rather  than  a  monotonic  series  of  test  speeds,  but  opted 
for  the  latter  because  it  would  more  systematically  reveal  effects  of  ‘self  adaptation’  to  the  test 
patterns  themselves.  By  repeating  the  whole  sequence  in  reverse,  the  degree  to  which  the 
resulting  velocity  tuning  curves  are  similar  confirms  the  validity  of  the  assumption  that 
adaptation  state  is  clamped  constant. 
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Figure  2:  HSNE  response  to  velocity  of  natural  images  animated  to  simulate  yaw  motion  in 
monotonic  series. 


2. 1.2. 3  Testing  with  a  larger  set  of  natural  images 

We  were  successful  in  obtaining  several  very  high  quality  recordings  of  long  duration  (over 
3  hours  in  one  case),  which  allowed  us  to  test  responses  to  many  different  image  sets.  In  the 
earlier  experiments  with  this  stimulus,  we  obtained  some  data  that  suggested  velocity 
constancy  (contrast  independence)  was  not  as  obvious  as  in  the  earlier  pilot  experiments,  at 
least  when  contrast  was  varied  by  re-scaling  the  luminance  values  (color  lookup  table 
compression)  within  an  image.  An  important  deficiency  in  all  of  these  experiments  was  that  our 
display  system,  with  a  maximum  contrast  ratio  of  around  250:1,  could  not  come  close  to 
matching  the  dynamic  range  of  'true'  natural  scenes,  so  that  simulations  of  natural  motion 
typically  have  lower  contrast  than  in  nature.  We  therefore  decided  to  test  responses  to  a  larger 
set  of  images  deliberately  selected  to  display  a  natural  variation  in  contrast  (and  possibly  in 
other  aspects  of  natural  image  statistics)  to  provide  a  test  for  the  hypothesis.  The  image  set 
developed  is  illustrated  below: 
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Figure  3:  9  panoramic  images  selected  for  testing  velocity  constancy:  Images  referred  to  in 
subsequent  text  as  follows,  A  “Hamlin”  a  contrast  scene  originally  captured  in  Hamlin 
woods,  near  Seattle  and  captured  from  the  position  of  a  hovering  fly  located  at  that  time. 
B:  “Barr  sun”:  captured  from  the  Bar-Smith  Library  lawns  at  the  University  of  Adelaide. 
C:  “close”:  captured  from  Botanic  park,  Adelaide  and  deliberately  including  close  detail  of 
a  plant  as  well  as  surrounding  scenery.  D:  “Distant”  the  same  scene  from  further  away 
from  the  plant  foliage.  E:  “Reception”:  the  reception  desk  area  of  the  Department  of 
Physiology,  University  of  Adelaide.  F:  “Linear  park”  taken  from  the  Adelaide  west 
parklands.  G:  “Glenelg”  taken  from  the  Glenelg  foreshore  near  Adelaide.  H:  “Gardens”  a 
scene  from  the  Botanic  Gardens  in  Adelaide  on  a  dull  day  to  deliberately  represent  a  low 
contrast  scene.  I:  “Lab  pan”  captured  in  the  O’Carroll  laboratory  temporary  space, 
University  of  Adelaide 


The  images  are  illustrated  in  fig.  3  in  an  order  that  matches  the  approximate  variation  in 
natural  contrast  of  this  set,  evaluated  by  a  simple  mean  absolute  deviation  measure.  By  this 
measure,  the  top  image  has  almost  4  times  the  overall  contrast  of  the  lower  two  (H,I).  We 
deliberately  included  two  artificial  enrvironments  amonst  the  image  set. 

A  significant  finding  was  the  confirmation  that  velocity  constancy  prevails  in  the  response 
of  HSNE  neurons  with  respect  to  different  natural  images;  i.e.,  HSNE  response  was  found  to  be 
velocity-dependent  but  invariant  when  the  animated  natural  images  were  varied.  This  is 
depicted  in  figure  4  below  for  the  full  set  of  images  in  1  recording  (blue  lines)  and  a  smaller  set 
in  a  second  recording  (orange).  The  degree  of  velocity  constancy  is  remarkable  considering  no 
post-experiment  normalization  has  been  applied. 
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Figure  4:  HSNE  response  to  natural  images  animated  to  simulate  yaw  motion.  Data  are 
obtained  from  two  different  cells.  An  ad-hoc  contrast  measure,  mean  absolute  deviation  as 
well  as  the  number  of  trials  over  which  results  are  averaged,  are  shown  for  each  image. 
Recording  by  A.  Straw  &  D.O’Carroll,  University  of  Adelaide 


Contrast  of  these  natural  images  varied  considerably,  as  indicated  in  the  figure.  However, 
another  significant  finding  is  that  artificial  contrast  reduction  (by  rescaling  the  range  of 
luminance  variations  while  maintaining  the  same  mean  luminance)  resulted  in  departure  from 
velocity  constancy  at  higher  levels  of  contrast  than  suggested  by  our  earlier  pilot  experiments: 


0  10  100  1000  3000 

Yaw  Velocity  (°/sec) 


Figure  5:  HSNE  response  to  moving  natural  images  under  artificial  reduction  in  image 
contrast.  Contrast  measures  are  relative  to  original  image  contrast 


The  reason  for  the  apparent  discrepancy  between  the  current  and  prior  results  was  at  this 
point  uncertain;  we  planned  to  standardize  our  methods  of  computing  image  contrasts  and  do  a 
detailed  comparison  in  the  future. 

2.1.3  Year  3 

In  our  experimental  work  this  period,  we  made  recordings  from  HS  cells  in  a  number  of 
different  animals,  using  the  wider  variety  of  images  illustrated  in  figure  3,  a  larger  set  of 
artificially  adjusted  contrasts,  and  a  greater  range  of  adapting  stimuli  than  in  prior  work,  in 
order  to  evaluate  the  conditions  under  which  velocity  constancy  holds.  These  recordings  were 
very  successful  —  we  set  a  lab  record  this  period  for  viability  of  an  HS  cell  following 
penetration  with  the  recording  electrode  (over  6  hours).  Analysis  from  these  experiments 
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confirmed  basic  properties,  in  particular  velocity  constancy  with  different  natural  images, 
observed  previously. 

2. 1.3.1  Saturating  nonlinearity  &  velocity  constancy  in  “ unadapted  ”  neurons 

Although  these  experiments  were  not  designed  specifically  to  test  the  mechanisms  driving 
the  gain  reduction  portion  of  adaptation,  preliminary  results  shed  some  light  on  the  property  of 
velocity  constancy.  While  it  was  assumed  that  adaptive  gain  reduction  /  gain  control  plays  a 
significant  role  in  velocity  constancy,  we  noted  other  mechanisms  that  could  contribute  to  this 
property,  that  may  be  in  operation  in  varying  degrees  in  the  experimental  results  we  obtained. 
One  such  mechanism  is  a  saturating  nonlinearity  in  the  EMD  correlator  outputs,  which  in 
combination  with  high  front-end  gain  results  in  hard  limiting  and  thus  relative  insensitivity  to 
changes  in  input  intensity  (saturation  of  these  signals  is  responsible  for  a  significant  decrease  in 
contrast  sensitivity  of  the  alternate  adaptive  model  under  consideration).  A  second  mechanism 
is  the  nonlinear  and  ‘normalizing’  nature  of  signal  summation  in  the  HS  cell  itself  (somewhat 
misleadingly  referred  to  as  ‘gain  control’  by  Borst  and  co-workers  in  prior  literature). 

The  relevant  results  have  to  do  with  the  response  of  the  HS  neurons  in  the  putative 
‘unadapted’  state  to  full-contrast  test  stimuli.  In  this  portion  of  the  experiments,  a  moving  test 
pattern  was  presented  after  ‘adaptation’  to  a  blank  screen  of  mean  luminance  for  at  least  5 
seconds,  with  the  pattern  appearing  in  motion  for  Is.  We  discarded  the  initial  50ms  transient 
then  averaged  membrane  potential  across  the  next  150ms.  As  always,  choosing  this  averaging 
'window'  was  problematic:  Because  the  images  were  very  high  in  contrast,  adaptation  was 
rapid,  so  the  curves  must  represent  at  least  a  partly  adapted  state.  (On  the  other  hand,  if  the 
averaging  'window'  were  shortened  to  further  minimize  this  'self  adaptation,  the  tuning  curves 
would  be  distorted  because  of  the  transient  behavior  of  the  system,  which  tends  to  bias  curves 
towards  higher  temporal  frequencies  and  hence  velocities.  Our  contrast  ramp  method 
(developed  in  the  mid  1990s  at  the  University  of  Cambridge)  would  have  provided  a  significant 
improvement  in  this  respect,  but  would  have  defeated  the  purpose  of  the  experiment,  which  is 
to  estimate  responses  at  the  full,  un-attenuated  image  contrast  in  order  to  recruit  the  full  range 
of  nonlinearities  operating  in  coding  such  images,  except  for  adaptation).  In  addition,  the 
variance  of  responses  due  to  variations  in  input  pattern  structure  /  phase  (‘pattern  noise’)  was 
high. 

In  spite  of  the  complicating  effect  of  these  difficulties  on  interpretation  of  the  results,  those 
results  suggested  a  surprisingly  strong  degree  of  velocity  constancy  even  in  the  unadapted  HS 
cell.  Figure  6  below  shows  unadapted  velocity  tuning  curves  averaged  across  several 
recordings  (4  cells  for  preferred  and  2  for  anti-preferred  direction)  where  the  initial  'phase'  of 
the  panoramic  image  was  altered  systematically  throughout  the  recordings  to  reduce  the 
influence  of  pattern  noise  (a  particular  problem  at  lower  speeds  in  this  experiment).  Prior  to  the 
application  of  the  adapting  stimuli  (i.e.,  after  presentation  of  blank  screens  of  mean  luminance), 
the  responses  of  the  HS  cells  to  the  test  stimuli  were  recorded  in  these  experiments. 
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Figure  6:  Mean  responses  of  ‘unadapted’  HS  cells  to  preferred  (left)  and  antipreferred 
(right)  direction  yaw  motion  of  natural  panoramic  stimuli.  Responses  to  three  different 
scenes  of  varying  contrast  are  depicted  and  annotated  by  the  name  assigned  to  each  as  in 
fig.  3  (except  that  mbsde=the  image  “Hamlin'’  referred  to  previously).  Responses  are 
averaged  over  several  cells  and  several  trials  with  differing  initial  phases.  Data  obtained  by 
T.  Rainsford  &  D.  O’Carroll 


These  initial  results  encompassed  three  images  (labeled  close ,  gardens  and  mbsde  = 
“Hamlin  ”  referred  to  earlier )  from  a  larger  set  that  was  presented  in  similar  experiments. 
These  three  images  represented  highest,  lowest  and  an  intermediate  contrast  in  the  set  by 
various  'traditional'  measures  of  contrast  in  natural  images  (e.g.  RMS  power,  mean  absolute 
deviation,  etc.).  During  this  period,  further  analysis  was  regarded  as  necessary  to  quantify  how 
their  'effective  contrasts’  differed  (i.e.,  contrast  with  respect  to  the  spatiotemporal  receptive 
field  of  HS  cells):  This  was  performed  in  concurrent  modeling  efforts. 

From  these  results,  we  concluded  that  mechanisms  for  velocity  constancy,  such  as  EMD 
subunit  saturation  and  the  nonlinear  summation  of  the  HS  cell  itself,  must  be  operational  even 
in  the  unadapted  motion  pathway.  This  naturally  raised  the  question  of  exactly  what  the  role  of 
contrast  gain  reduction  is  in  this  system  (noting  that  its  presence  is  indisputable  based  on  the 
results  of  contrast  sensitivity  experiments  before  and  after  adaptation).  It  also  showed  that 
global  mechanisms  for  velocity  constancy  in  any  experiment  are  likely  to  seriously  confound 
efforts  to  infer  the  effects  of  contrast  gain  adjustment.  Such  global  mechanisms  have  the  effect 
of  minimizing  the  results  of  changes  in  the  gain  at  the  front  end  of  the  system. 

2. 1.3. 2  Test-adapt-test  stimuli 

In  further  experimental  work,  we  continued  to  make  recordings  from  HS  cells  in  specimens 
of  Eristalis,  and  examined  adapted  as  well  as  unadapted  responses  to  motion.  Experiments 
continued  with  the  three  images  {close,  gardens  and  mbsde )  used  in  the  previous  experiments 
which  suggested  substantial  'velocity  constancy'  even  in  the  unadapted  system.  Using  four 
different  contrast  measures,  gardens  has  between  one-quarter  and  one-third  the  contrast  of 
mbsde  (=  ” Hamlin  ”). 

Further  tests  were  done  with  the  interleaved  test-adapt  protocol.  The  adapting  stimulus  was 
the  same  image  that  was  presented  during  the  test  phases  and  was  always  presented  at  a 
(maximal)  contrast  of  1  and  at  a  speed  of  100  degrees  per  second.  In  the  unadapted  cases,  the 
adapting  stimulus  was  a  blank  screen  of  mean  luminance.  During  the  test  phases,  however,  the 
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contrast  of  the  image  was  varied  artificially  by  compressing  the  range  of  variation  around  the 
mean  luminance  of  the  raw  image.  This  variable  contrast  test  was  presented  before  and  after  the 
adapting  stimulus,  as  in  our  earlier  published  work  on  contrast  gain  reduction  (Harris  et  al 
2000). 

Figure  7  &  8  below  completely  summarize  the  results  of  this  set  of  experiments.  These  new 
data  were  consistent  with  the  presence  of  saturating  nonlinearities  in  the  system,  supporting  the 
hypothesis  that  saturation  may  play  a  role  in  velocity  constancy.  The  fact  that  the  unadapted 
curves  (red)  were  broad  and  'flat-topped’  (platykurtic)  relative  to  the  adapted  curves  (green) 
suggested  that  saturation  is  more  prevalent  in  the  unadapted  system  (consistent  with  the 
hypothesis  that  adaptation  modifies  the  system  gain  at  the  front  end  of  the  EMDs).  In  addition 
to  the  difference  in  shape  of  the  tuning  curves,  the  maximal  response  in  the  adapted  curves  was 
reached  at  higher  velocities  than  in  the  unadapted.  The  adapted  responses  decreased  in 
amplitude  more  rapidly  with  decreasing  contrast  than  the  unadapted,  which  was  also  consistent 
with  low  contrast  gain  in  the  adapted  state. 

The  unadapted  velocity  response  curves  at  a  contrast  of  1  were  remarkably  similar  for  all 
images.  However,  the  curves  at  various  artificially  reduced  contrasts,  for  each  individual 
image,  varied  greatly  in  their  amplitude  (compare  the  3  highest  contrasts  c=l,  0.75  and  0.5), 
despite  the  fact  that  the  range  of  contrasts  was  less  than  the  variations  among  the  three  images 
themselves  at  their  maximal  or  'natural'  contrast.  This  we  regarded  as  very  difficult  to  explain 
based  on  any  hypotheses  at  the  time.  The  only  difference  between  the  artificially  reduced 
contrast  and  that  seen  in  the  natural  variation  of  the  un-attenuated  images  was  the  range  of 
luminance  values  for  individual  pixels:  in  an  artificially  reduced  image,  the  histogram  was 
compressed  about  the  mean  luminance.  This  might  be  expected  to  lead  to  increased  gain  for 
such  images,  due  to  contrast  adaptation  of  early  visual  neurons  such  as  photoreceptors  and 
LMCs.  Yet,  the  observed  responses  were  weaker  than  their  counterparts  for  natural  images 
with  similarly  low  contrast,  which  however  still  exercise  the  full  dynamic  range  of  the 
grayscale  image  (i.e.  8  bits,  or  luminance  values  from  0-255). 
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Figure  7:  Preferred  direction  velocity  tuning  curves  from  HS  cells  with  the  system  in  the 
unadapted  (red)  and  adapted  (green)  states.  Responses  to  three  different  images  are 
depicted.  Contrasts  (relative  to  original  image  contrast)  are  varied  parametrically. 
Responses  are  averaged  over  several  cells  and  several  trials  with  differing  initial  phases. 
Abscissa  represents  speed  in  degrees  per  second  in  log  scale,  and  ordinate,  membrane 
potential  relative  to  rest  in  millivolts.  Data  obtained  by  T.  Rainsford  &  D.  O’Carroll. 
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Figure  8:  Antipreferred  direction  velocity  tuning  curves  from  HS  cells  with  the  system  in 
the  unadapted  (red)  and  adapted  (green)  states.  Responses  to  three  different  images  are 
depicted.  Contrasts  (relative  to  original  image  contrast)  are  varied  parametrically. 
Responses  are  averaged  over  several  cells  and  several  trials  with  differing  initial  phases. 
Abscissa  represents  speed  in  degrees  per  second  in  log  scale,  and  ordinate,  membrane 
potential  relative  to  rest  in  millivolts.  Data  obtained  by  T.  Rainsford  &  D.  O’Carroll. 

We  subsequently  continued  to  make  recordings  from  HS  cells  in  specimens  of  Eristalis, 
and  ran  initial  experiments  in  which  small-signal  test  or  probe  stimuli  were  used  in  a  effort  to 
evaluate  the  front-end  gain  in  the  motion  system,  and  its  dependence  upon  parameters  of  the 
adapting  stimulus  (i.e.,  contrast  and  speed  of  motion).  The  adapting  stimulus  was  large-signal 
(i.e.,  full  contrast  moving  image  covering  much  of  the  receptive  field  of  the  neuron  under  test), 
and  was  interleaved  with  test  stimuli  in  which  contrast  was  reduced  by  compression  of  intensity 
variations  about  the  mean  luminance  of  the  scene. 

Experiments  were  carried  out  using  the  same  'standard'  set  of  3  natural  panoramic  images 
(‘high’,  ‘medium’,  and  'low'  contrast)  used  previously,  as  adapting  stimuli  (at  full  contrast  and 
various  speeds),  and  as  probe  stimuli  (with  artificially  compressed  (relative)  contrasts  of  0.1 
and  0.12,  and  at  the  near-optimal  speed  of  1007s).  Preferred-  and  antipreferred-direction 
adapting  stimuli  were  used.  The  intention  was  to  use  the  difference  in  response  to  the  0.1  and 
0.12  probe  contrasts  as  the  measure  of  gain.  Unfortunately,  these  two  contrasts  proved  to  be  too 
weak  in  the  adapted  state,  so  that  the  results  were  too  noisy  to  make  this  analysis  meaningful. 
However,  to  convey  a  picture  of  the  small-signal  response,  the  results  of  the  experiments  (three 
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scenes  and  two  reduced  contrast  values)  were  simply  averaged  together  for  each  direction. 
Results  are  depicted  in  figure  9  below: 


Figure  9:  Averaged  HS  cell  responses  to  low-contrast  test  stimuli  following  adaptation. 

Results  for  both  preferred  (black)  and  antipreferred  (red)  direction  of  motion  are  shown. 

Control  response  of  the  motion-adapted  system  to  full-contrast  stimuli  is  shown  in  blue. 

The  dashed  horizontal  line  represents  the  resting  potential.  Pattern  Velocity  on  the 
abscissa  refers  to  the  velocity  of  the  adapting  stimulus  (probe  stimuli  were  all  at  1007s). 

Error  bars  indicate  standard  error.  Data  were  obtained  by  T.  Rainsford  &  D.  O’Carroll. 

These  curves  clearly  show  an  inverse  relation  between  probe  response  and  the  full-contrast 
tuning  curves,  which  is  consistent  with  an  adaptive  gain  reduction  that  is  dependent  upon 
system  output,  and  thus  with  the  presence  of  a  feedback  mechanism.  However,  the  results  were 
not  entirely  conclusive  because  they  were  confounded  with  changes  in  the  resting  potential: 
while  the  preferred-direction  responses  were  consistent  with  a  shift  downward  due  to  the 
‘waterfall  effect’,  the  anti-preferred  direction  responses  (in  which  no  after-potential  effects  are 
expected)  were  above  the  resting  potential  for  many  of  the  points  on  the  curve.  In  addition, 
dependence  of  the  small-signal  responses  on  adapting  pattern  contrast  was  not  tested  directly  in 
these  experiments,  so  definitive  answers  to  the  question  of  what  drives  contrast  gain  reduction 
in  adaptation  awaits  further  experiments. 

Due  to  lack  of  availability  of  Eristalis  during  the  onset  of  winter  no  further  progress  was 
made  on  this  species  (our  primary  experimental  preparation  for  the  project).  In  experiments  on 
an  alternative,  the  blowfly  Calliphora,  most  recordings  were  made  from  VS  and  CH  cells 
rather  than  HS  cells,  and  involved  the  test-adapt-test  protocol.  These  data  were  of  poor  quality 
and  largely  unanalyzed  during  the  period,  but  suggested  similar  adaptive  behavior  and  tuning  in 
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their  responses  with  respect  to  preferred  direction  stimuli  (vertical  motion  in  this  case),  to  what 
had  been  observed  in  HS  cells. 

2. 1.3. 3  Small  signal  contrast  gain 

In  our  prior  attempts  to  extract  the  small-signal  contrast  gain  of  HS  cells  using  low-contrast 
probe  stimulus,  we  mainly  demonstrated  the  difficulty  of  estimating  appropriate  test  contrast 
values  to  elicit  responses  both  above  the  noise  floor  and  below  any  saturating  effects.  In 
addition,  the  confounding  effects  of  possible  shifts  in  the  ‘resting’  or  baseline  membrane 
potential  compounded  the  difficulty.  Because  of  these  unsuccessful  experiments,  we  decided  to 
adopt  a  more  involved  method  in  which  a  contrast  sensitivity  curve  was  generated  for  each 
state  of  adaptation  to  be  investigated,  and  worked  on  both  the  details  of  the  issues  involved 
with  this  approach  and  the  methodology  to  implement  it.  Our  intent  under  this  protocol  was  to 
apply  a  standard  or  canonical  form  of  probe  stimulus  (such  as  an  optimal  frequency  grating)  in 
all  such  experiments,  so  that  all  measurements  of  contrast  gain  so  obtained  would  be  certain  to 
be  comparable.  This  required  interleaving  of  the  probe  stimulus  with  the  various  adapting 
stimuli,  which  necessitated  some  further  programming  efforts  in  the  Vision  Egg  stimulus 
generating  system. 

We  also  analyzed  a  large  volume  of  data  previously  obtained  from  Eristalis  which 
confirmed  tentative  conclusions  reached  from  pilot  experiments:  following  adaptation  to 
natural  images,  which  may  vary  significantly  in  contrast,  a  high  degree  of  velocity  constancy 
was  observed  in  the  cell  response.  This  conclusion  was  found  to  apply  even  when  the  velocity 
of  the  adapting  stimulus  was  varied  over  a  wide  range. 

These  experiments  were  carried  out  using  the  'standard'  test-adapt-test  protocol:  an  adapting 
stimulus,  consisting  of  a  panoramic  image  at  full  natural  contrast,  was  presented  at  a  fixed 
speed  and  interleaved  with  test  stimuli  of  the  same  image  at  varying  speeds.  However,  the 
dimension  of  the  adapting  stimulus  speed  (previously  limited  to  a  few  different  values)  was 
thoroughly  explored  in  a  large  set  of  such  experiments.  These  were  repeated  for  both  preferred 
and  antipreferred  directions. 

A  summary  is  presented  below  in  figure  10  in  which  individual  response  curves  for 
adapting  stimuli  at  different  velocities  are  shown  as  smooth  unbroken  lines,  and  their  averages 
as  heavy  lines  in  which  markers  indicate  individual  data  points. 
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Figure  10:  HS  cell  responses  to  test  stimuli  following  adaptation  to  a  wide  variety  of 
adapting  speeds.  Individual  response  curves  (smooth  unbroken  lines)  are  each  for  a 
particular  adapting  stimuli  velocity  varied  in  the  range  5-2000  deg/s.  The  heavy  line  (with 
error  bars)  is  the  mean  for  all  data.  Results  are  shown  for  low-contrast  (‘gardens’)  and 
high-contrast  (‘mbsde’)  natural  images,  which  has  4  times  the  contrast  of  other  (as 
measured  by  several  different  contrast  metrics).  Data  are  shown  for  both  preferred  (a) 
and  anti-preferred  (b)  directions.  Data  were  obtained  by  T.  Rainsford  &  D.  O’Carroll 


The  degree  of  velocity  constancy  under  such  a  wide  range  of  adapting  conditions,  even  for 
velocities  which  induce  relatively  weak  responses  in  the  target  cell,  is  impressive.  Further 
analysis  of  the  same  data  set  showed  that  deviations  from  ‘perfect’  velocity  constancy  occurred 
at  the  limits  of  the  range  of  speeds  over  which  the  HS  cell  responds.  In  these  cases,  responses 
to  the  constant  probe  stimuli  were  enhanced,  in  comparison  with  other  adapting  stimuli.  This 
supported  the  notion  that  strongest  adaptation  effects  are  recruited  by  stimuli  that  drive  the 
EMDs  maximally. 

We  subsequently  obtained  a  number  of  additional  recordings  from  HS  cells  in  Eristalis. 
These  recordings  were  done  primarily  to  obtain  additional  supporting  data  for  the  velocity 
constancy  experiments  in  adapted  and  unadapted  HS  cells,  to  increasing  sample  sizes  to  a  level 
acceptable  for  a  published  paper.  We  also  began  work  on  a  manuscript  for  publication  that  will 
include  these  results  (This  paper,  Straw,  Rainsford  and  O’ Carroll,  has  subsequently  been 
submitted  but  has  not  yet  been  accepted  for  publication). 
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2. 1.3. 4  New  experimental  Protocol  for  evaluating  contrast  gain  in  adapted  neurons 

Due  to  the  difficulties  with  evaluating  contrast  gain  with  the  ‘contrast  sensitivity’ 
experiments  discussed  in  prior  periods,  we  began  modifying  the  ‘Vision  Egg’  stimulus 
generation  and  display  system  to  permit  more  elaborate  protocols  that  will  allow  unambiguous 
evaluation  of  small-signal  contrast  gain.  These  modifications  included:  a)  implementing  the 
ability  to  switch  textures  in  the  panoramic  image  stimulus.  This  allows  a  stereotyped  test 
(probe)  stimulus  such  as  an  optimal  frequency  sinusoidal  grating  to  be  used  while  adaptation 
occurs  with  a  natural  image;  (b)  Ability  to  window  the  panoramic  stimulus  to  reduced  angular 
extent  (with  Gaussian  window  to  'soften’  the  stimulus  edges).  This  allows  testing  of  the  theory 
that  the  constancy  observed  for  unadapted  velocity  tuning  derives  in  part  from  non-linearity  in 
spatial  integration  (i.e.  local  saturation,  or  the  Borst  ‘gain  control’  mechanism).  It  also  permits 
testing  of  the  spatial  locality  of  the  possible  mechanisms  for  motion  adaptation. 

We  implemented  this  probe-adapt-probe  experiment  protocol  in  order  to  quantify  the  gain 
change  induced  by  an  adapting  stimulus.  The  stimulus  was  implemented  as  a  modification  of  a 
VisionEgg  stimulus  that  projects  a  textured  'spinning  drum'  onto  the  monitor.  In  previous 
experiments  of  this  type,  we  rendered  a  single  texture  in  order  to  simulate  motion  at  different 
speeds.  In  the  new  stimulus,  we  cache  two  textures,  one  as  the  adapting  stimulus  and  one  as  the 
probe. 

After  several  pilot  experiments,  we  settled  on  a  sinusoidal  grating  stimulus  as  a  canonical 
'probe'  texture.  The  sinusoid  has  an  integer  number  (36)  of  cycles  in  360  degrees,  so  that  when 
mapped  onto  the  cylinder  it  can  be  moved  continuously  without  edge  effects.  The  spatial  period 
(10  degrees)  is  close  to  optimal  for  HS  neurons  when  the  pattern  is  presented  frontally,  and  we 
settled  on  a  velocity  (ca.  80  degrees/s)  that  induces  optimal  temporal  frequency  (8  Hz)  for  male 
HS  neurons,  based  on  earlier  work  (O’Carroll  et  al  1997).  This  sinusoidal  stimulus  lacks  the 
'pattern  noise'  associated  with  low  contrast  presentation  of  natural  images.  Because  it  is  an 
'optimal'  stimulus,  we  were  therefore  able  to  estimate  response  magnitude  at  low  contrasts  with 
relatively  little  signal  averaging. 

In  initial  experiments  we  obtained  recordings  from  several  HS  neurons  and  an  interesting 
VS  (vertical  system)  type  neuron.  We  presented  the  probe  stimulus  for  1  second  before  and 
after  presentation  of  an  adapting  stimulus  (in  this  case  our  default  high  contrast  panoramic 
natural  image),  and  then  varied  the  contrast  of  the  test  pattern  over  a  large  range  in  order  to  (a) 
estimate  the  unadapted  contrast/response  function  (i.e.  in  response  to  the  first  test),  and  (b)  to 
evaluate  the  same  curve  in  the  motion-adapted  state  for  purposes  of  establishing  ideal  contrast 
values  to  select  in  further  experiments.  In  the  VS  cell,  the  contrast  threshold  was  very  low: 
significant  responses  were  evoked  by  contrasts  as  low  as  1%  for  the  canonical  test  pattern 
before  adaptation.  We  subsequently  presented  the  stimulus  at  pairs  of  probe  contrasts,  in  each 
case  differing  by  10%  (e.g.  1%  and  1.1%).  Even  though  the  system  has  a  non-linear  'gain'  with 
respect  to  contrast,  the  small  difference  between  these  probe  contrasts  allowed  us  to  define 
contrast  gain  within  an  approximately  ‘linear’  part  of  the  response  range: 

Gc=dR/dC 

In  fact,  because  of  the  quadratic  non-linearity  involved  in  motion  correlation,  the  square  root  of 
Gc  would  be  a  measure  of  the  ‘input  gain’  of  the  system.  By  measuring  the  change  in  Gc  before 
and  after  adaptation,  we  were  able  to  evaluate  a  ‘tuning  curve’  for  the  parameters  that  drive 
adaptation. 

The  biggest  problem  with  this  technique  was  the  need  to  know,  in  advance,  the  approximate 
range  of  contrasts  that  are  most  useful.  In  our  pilot  experiments  we  explored  contrast  pairs  in 
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the  range  from  1  or  1.1%  to  10  or  1 1%  contrast  for  the  sinusoid.  In  one  recording  we  held  the 
cell  long  enough  to  do  this  for  several  different  adapting  stimulus  speeds.  These  pilot  data  were 
used  to  plan  further  experiments  that  were  optimized  for  the  task,  and  thus  to  obtain  complete 
tuning  curves. 

We  further  elaborated  our  protocol  for  evaluation  of  contrast  gain  based  on  the  responses  of 
wide-field  motion-sensitive  neurons,  as  well  as  a  method  of  analysis  that  we  believe  will  give  a 
robust  indicator  of  shifts  on  contrast  sensitivity  based  on  a  relatively  limited  number  of 
response  data.  By  limiting  the  number  of  data  required  from  each  experimental  trial,  this 
approach  was  expected  to  permit  experiments  covering  a  large  range  of  adapting  conditions 
during  the  typical  period  of  viability  of  lobula  plate  cells  following  electrode  penetration.  We 
obtained  experimental  data  and  performed  analyses  for  several  neurons  using  these  methods. 

The  experimental  procedure  consists  of  two  parts.  First  is  a  control  experiment,  where  a 
canonical  probe  stimulus  (a  grating  with  optimal  spatial  frequency  moving  in  the  preferred 
direction  for  the  neuron)  is  presented  at  a  series  of  contrasts  from  below  threshold  (i.e. 
contrasts  that  produce  no  measurable  response)  to  the  maximum  that  can  be  generated  (=1.0  for 
a  sinusoid).  Between  each  stimulus,  the  neuron  is  unstimulated,  and  the  insect  views  a  blank 
screen  of  mean  luminance  for  at  least  8s.  We  then  plot  response  (membrane  potential  for  a 
graded  neuron  like  HS,  or  1/inter-spike  interval  for  a  neuron  that  generates  action  potentials, 
such  as  VS)  as  a  function  of  the  contrast  of  the  stimulus.  On  the  assumption  that  this  ‘rest’ 
period  negates  the  effect  of  adaptation  due  to  the  probe  stimuli,  we  called  the  resulting  contrast 
sensitivity  function  (CSF)  the  ‘unadapted’  CSF. 

We  fit  the  unadapted  CSF  with  a  Weibull  function  of  the  following  form,  using  non-linear 
function  minimization  (Nelder-Mead  Simplex  method): 

f(x)=gain*(l -(exp(-((x. /alpha). Abeta)))); 

An  example  for  an  HS  neuron  is  shown  below  in  figure  1 1 : 
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Pattern  contrast  (test  stimuli) 

Figure  11:  Contrast  Sensitivity  function  for  an  unadapted  HS  neuron.  Data  obtained  by  T. 

Rainsford  &  D.  O’Carroll 

In  agreement  with  our  earlier  results  evaluating  contrast  sensitivity  in  these  neurons  (O’Carroll 
et  al  1997, Harris  et  al  2000),  we  found  that  the  unadapted  CSF  for  both  HS  and  VS  neurons 
indicate  they  respond  down  to  very  low  contrasts  of  just  1-2  %. 

The  aim  of  this  control  experiment  is  not  only  to  determine  the  shape  of  the  unadapted 
CSF,  but  also  the  saturation  response  level  for  the  neuron.  In  particular  for  HS  neurons,  the 
quality  of  individual  recordings  varies,  due  to  the  difficulty  of  inserting  the  glass 
microelectrode  into  the  neuron  without  damage  to  the  surrounding  membrane  (the  neuron 
diameter  is  ca.  1  p,m  -  2  pm,  while  our  microelectrode  tips  are  ca.  50nm  -  lOOnm).  Hence,  the 
gain  parameter  in  the  above  equation  does  not  correspond  to  the  contrast  gain  that  we  are  trying 
to  estimate  with  this  experiment,  but  rather  a  variable  that  accounts  for  the  range  of  membrane 
potentials  over  which  an  individual  neuron  can  depolarize  (or  maximum  firing  rate  for  a 
spiking  neuron).  We  can  then  quantify  the  contrast  sensitivity  of  the  neuron  by  solving  the 
fitted  function  for  response  level  criteria  (10%  and  50%  of  the  maximum  being  illustrated  in 
the  curve  above).  These  threshold  contrasts  give  an  estimate  of  the  gain  of  the  CSF.  Since 
lower  gain  results  in  a  right-shift  in  the  CSF,  it  is  convenient  to  take  1/threshold  as  an  estimate 
of  the  contrast  sensitivity  of  the  system,  CS,  where  CS  is  directly  proportional  to  contrast  gain. 

The  next  step  in  the  protocol  is  a  more  demanding  probe-adapt-probe  sequence,  where 
probe  stimulus  is  similar  to  that  in  the  control  experiment  is  used,  but  with  several  seconds  of 
adaptation  to  a  variable  stimulus  before  the  second  probe  in  each  case.  After  each  sequence  is 
presented,  the  neuron  is  again  allowed  to  recover  by  viewing  a  blank  screen  for  several 
seconds.  We  compare  the  CSF  obtained  for  the  neuron  in  the  first  probe  of  each  sequence  with 
the  control  to  check  that  the  rest  interval  is  adequate.  We  found  that  CS  solved  for  such  data  is 
indistinguishable  from  the  control  if  we  use  rest  periods  greater  than  5s.  For  these  data,  we 
estimate  the  response  to  both  probe  stimuli  and  use  Simplex  minimization  to  find  the  best  fit  to 
the  data  of  models  for  both  unadapted  and  adapted  responses: 
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f(x)  =gain  *( 1  -(exp(-( (x./alpha). A beta )))); 
f>  (x)  =gain  *(l-( exp(-  ((x.  /alpha 2)  .Abeta2))))', 

where  the  subscript  “2”  indicates  the  function  that  is  applied  to  the  adapted  response.  Note  that 
this  double-Weibull  model  uses  a  single  gain  parameter  for  both  functions,  on  the  assumption 
that  the  basic  recording  properties  of  the  cell  do  not  change. 

We  then  compute  two  sum-square  error  terms,  q,  q2,  as  the  difference  between  f(x),  f2(x) 
and  the  actual  data  measured  for  the  cell  to  the  two  probe  stimuli: 

q=I(fy-f(x))/2); 

q2=I  ( (y2-(f2(x) + offset)). A2) ; 

where  y  and y2  indicate  the  unadapted-  and  adapted-state  experimental  data,  respectively.  Note 
that  the  second  term  includes  an  offset  parameter,  which  allows  us  to  account  for  DC 
membrane  potential  changes  that  we  frequently  observe  following  powerful  adapting  stimuli 
(the  so-called  waterfall  component  of  adaptation,  or  motion  after-effects).  The  sum  of  q  and  q2 
is  then  minimized.  The  resulting  minimization  thus  uses  5  parameters 
(alpha, alpha2, beta, beta2,  offset).  Note  also  that  the  gain  used  in  the  model  fits  is  not  varied 
during  minimization,  being  fixed  to  the  value  obtained  for  the  control  experiment. 

An  example  of  this  double-Weibull  fit  is  shown  below  in  figure  12,  where  the  adapting 
stimulus  was  a  high  contrast  panoramic  natural  image  moving  at  100  deg/s  in  the  preferred 
direction: 


Figure  12:  Fits  to  contrast  sensitivity  functions  using  data  obtained  from  the  adapted  (red) 
and  unadapted  (black)  states.  Data  obtained  by  T.  Rainsford  &  D.O’Carroll 
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Note  that  with  the  addition  of  the  offset  back  into  f2(x)  (solid  red  lines  in  above  plots)  the 
model  provided  a  convincing  fit  to  the  data  despite  the  paucity  of  points  sampled.  This  reflects 
the  benefit  of  fixing  the  gain  term  based  on  a  control  experiment.  Providing  the  recording 
remains  healthy,  the  Weibull  lit  was  found  to  be  extremely  robust  when  applied  to  such  data. 

We  then  solve  the  fitted  Weibull  functions  for  precise  response  threshold  criteria,  either 
with  or  without  the  DC  offset  term.  This  allows  determination  of  the  shift  in  contrast  sensitivity 
due  to  both  gain  reduction  and  the  DC  offset  component  (waterfall  effect).  As  our  pilot  data 
show,  we  were  able  to  obtain  good  quality  fits  with  as  few  as  5  sampled  points  in  the  contrast 
domain,  greatly  reducing  the  experiment  duration  compared  with  that  in  previous  work.  This 
made  it  feasible  to  vary  the  adapting  stimulus  over  a  limited  range  within  a  typical  experiment 
duration  (10  minutes- 1  hour). 

We  were  subsequently  successful  in  applying  this  method  to  2  further  recorded  neurons, 
one  a  spiking  VS  cell,  and  the  other  an  HS  neuron.  Typical  fits  to  the  VS  data  are  shown  for  3 
different  adapting  speeds  in  Figure  13 
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VS  Neuron,  28/1 1/2003 
20  deg/s,  preferred  direction  adaptation 


1 00  deg/s,  preferred  direction  adaptation 


2000  deg/s,  preferred  direction  adaptation 


100  deg/s,  anti-preferred  direction  adaptation 


Figure  13:  Double  Weibull  function  fits  and  50%  response  criteria  computation  for  a 
contrast  sensitivity  of  a  VS  neuron  for  several  adapting  stimuli.  Data  obtained  by  T. 
Rainsford  &  D.O’Carroll 


In  this  case,  we  were  experimenting  to  find  the  optimal  (subset)  set  of  contrasts  at  which  to 
evaluate  the  adapted  response  in  each  case  (hence  the  variable  number  of  points  for  each 
graph).  These  data  showed  that  adapting  speeds  in  the  vicinity  of  the  response  optimum  for  the 
cell  (100-300  deg/s)  produced  the  most  convincing  rightward  shift  in  the  CSF  following 
adaptation  (red  data).  Note  also  that  even  when  very  few  points  were  sampled  (e.g.  at  2000 
deg/s),  the  solved  contrast  sensitivity  for  the  unadapted  response  (black  lines)  was  similar  to 
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that  obtained  in  control  experiments.  Interestingly,  the  effect  of  reversing  the  pattern  direction 
during  the  adapting  stimulus  produced  a  similar  right-shift  in  the  CSF,  but  appeared  to  have  a 
marked  effect  on  the  shape  of  the  CSF  in  this  spiking  neuron.  We  did  not  observe  this  in  our 
earlier  recordings  from  HS  neurons,  and  it  could  have  been  due  to  additional  non-linearity 
associated  with  the  spike  generating  mechanism:  This  throws  into  some  doubt  any  conclusions 
based  solely  on  data  obtained  for  spiking  responses  only  (as  is  always  the  case  for  extracellular 
recordings  from  cells  like  HI,  commonly  used  as  models  for  studying  insect  visual  processing). 
It  also  emphasizes  the  strengths  of  the  intracellular  recording  technique,  where  we  can  analyze 
membrane  potential  independent  of  spiking  responses. 

At  this  point,  we  were  able  to  use  data  of  this  form  to  graph  contrast  sensitivity  (CS) 
following  different  adapting  pattern  speeds.  An  example  for  the  above  neuron  is  shown  below 
in  figure  14. 


VS  Neuron 


Adapting  speed  (°/s) 


Figure  14:  Contrast  sensitivity  of  VS  neuron  as  a  function  of  adapting  stimulus  speed. 

Data  for  this  neuron  illustrates  3  important  points:  (1)  The  unadapted  contrast  sensitivity 
(open  symbols)  estimated  in  each  short  sequence  of  the  probe-adapt-probe  stimulus  was  in 
good  agreement  with  the  control  experiment  data  (dashed  line).  (2)  Direction  reversal  of  the 
adapting  stimulus  produced  similar  shift  in  contrast  sensitivity  (red  data  point)  to  the  preferred 
direction  stimulus,  indicating  that  the  gain  control  mechanism  is  not  inherently  direction 
selective.  (3)  The  strongest  adaptation  (lowest  gain  following  adaptation)  was  observed  when 
the  adapting  stimulus  is  close  to  the  optimal  speed  for  the  neuron.  The  latter  finding  generally 
supports  the  notion  that  the  adaptation  process  is  driven  by  feedback  from  motion  detectors,  but 
this  conclusion  is  somewhat  contradicted  by  the  fact  that  the  overall  ‘tuning’  of  adaptation  is 
very  broad,  with  powerful  adaptation  still  evident  at  the  highest  speeds  used  in  this  experiment. 
By  contrast,  the  HS  or  VS  neurons  gives  little  or  no  sustained,  measurable  response  to  such 
stimuli. 

We  evaluated  the  effect  of  varying  the  adapting  contrast  during  a  probe-test-probe 
sequence,  using  the  Weibull  curve  fit  experiments  outlined  above.  We  obtained  partial  data  sets 
from  several  HS  neurons,  and  more  complete  data  sets  in  2  cases.  Figure  15  shows  the  results 
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of  curve  fits  to  the  data  obtained  from  the  two  cells,  using  two  different  images  as  the  adapting 
pattern.  Each  point  is  the  mean  of  all  4  conditions  ±  standard  error. 

Although  we  considered  several  metrics  for  evaluating  the  gain  reduction  in  these  data  sets, 
the  simplest  (and  a  theoretically  sound)  method  which  was  eventually  put  into  use  was  to 
evaluate  a  response  threshold  in  the  adapted  state.  For  the  data  below,  this  is  the  probe  stimulus 
contrast  required  to  evoke  a  50%  maximal  response  from  the  neuron,  determined  from  the 
Weibull  curve  fit  to  the  contrast/response  function.  The  upper  plots  in  fig.  15  show  this  metric. 
As  gain  was  reduced  by  adaptation,  a  higher  stimulus  contrast  was  needed  to  evoke  this 
response.  The  threshold  contrast  is  inversely  proportional  to  contrast  gain  of  the  system. 

This  experiment  is  technically  challenging  and  very  demanding  time-wise,  in  that  each 
point  on  the  resulting  curves  requires  an  independent  experiment  in  which  contrast  of  the  probe 
stimulus  is  varied  in  sequential  presentations.  Because  it  is  vital  that  the  adaptation  state  be 
dominated  by  the  adapting  stimulus,  rather  than  the  probe  itself,  we  needed  to  employ  (a)  long 
adapting  times  and  (b)  long  ‘rest’  intervals  between  experiments.  In  the  data  obtained  during 
this  period,  we  used  a  total  duty  cycle  of  16  seconds.  Despite  these  measures  (which  led  to 
experiment  durations  of  >20  minutes),  analysis  of  the  response  to  the  first  probe  in  each  case 
(‘unadapted’  in  fig.  15  below)  showed  a  slight  but  worrying  upward  trend  in  the  threshold 
contrast  for  the  control  experiment.  This  suggested  that  some  of  effect  of  adaptation  evoked 
during  each  probe-test-probe  sequence  lasted  until  the  first  probe  stimulus  of  the  next,  at  least 
when  the  adapting  contrast  was  high.  Consequently,  in  experiments  performed  after  this  period, 
we  would  increase  the  interval  between  experiments  in  an  attempt  to  abolish  this  effect. 

In  spite  of  these  issues,  the  result  of  these  experiments  was  very  clear:  Adaptation  was 
found  to  be  strongly  dependent  on  contrast.  Indeed,  the  threshold  contrast  appears  to  increase 
linearly  up  to  the  maximum  (unattenuated)  contrast  of  the  natural  scenes  used  as  adapting 
stimuli  in  this  instance.  The  latter  observation  is  particularly  interesting  because  earlier 
experiments  on  the  unadapted  system  showed  that  these  higher  contrasts  lead  to  saturation  of 
the  response.  The  continued  linear  (and  possibly  even  supra-linear)  contrast  dependence  of  gain 
reduction  could  be  taken  as  evidence  that  the  adaptation  mechanism  incorporates  an  expansive 
non-linearity.  Since  most  other  non-linearities  observed  in  the  motion  pathway  are  compressive 
(e.g.  logarithmic  luminance  coding,  saturation),  these  findings  provide  strong  support  for  our 
computational  models,  where  gain  control  is  either  gated  by  feedback  from  EMD  outputs  or 
directly  modulated  by  it:  Tinder  these  conditions  the  non-linear  (multiplicative)  correlation 
mechanism  explains  the  dramatic  onset  in  adaptation  at  higher  contrasts. 

The  method  we  employed  also  allowed  us  to  address  a  second  question:  Are  the  so-called 
‘waterfall  effect’  and  gain  reduction  components  of  adaptation  similarly  driven  by  contrast? 
Both  components  lead  to  a  net  decrease  in  response  at  a  given  contrast.  The  magnitude  of  the 
waterfall  effect,  though,  is  quantified  by  the  overall  DC  offset  in  the  Weibull  fits  to  adapted 
curves.  The  lower  figure  shows  that  this  waterfall  component  increases  in  magnitude  as 
contrast  increases,  again  in  a  near-linear  manner  for  larger  adapting  contrasts.  By  subtracting 
this  component  away  from  the  adapted  curves,  we  were  able  to  determine  the  gain  reduction 
independent  of  it  (filled  triangle  symbols  in  the  upper  plots).  The  data  show  that  both 
components  of  adaptation  are  similarly  dependent  on  contrast. 
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Gain  reduction  effect 


Waterfall  effect 


Adapling  ccntrail  Adapting  COPiraBt 


Figure  15:  Contrast  threshold  (top)  and  post- hyperpolarization  (bottom)  as  a  function  of 
adapting  stimulus  contrast.  Data  are  averaged  over  two  HS  cells  and  two  different 
(contrast-attenuated)  natural  images.  The  abscissa  is  linear  at  left  and  logarithmic  at  right. 

Data  obtained  by  T.  Rainsford  &  D.O’Carroll. 

The  data  shown  above  were  pooled  from  both  images  used  as  adapting  stimuli  -  the  lowest 
and  highest  contrast  images  from  the  set  that  we  used  in  the  experimental  program  during  this 
stage  of  the  work.  At  this  point,  we  did  not  have  sufficient  data  to  differentiate  the  adaptation 
generated  by  these  two  images,  which  have  intrinsic  statistical  differences  (particularly  in  the 
contrast  domain),  although  both  were  taken  from  natural  habitat  that  the  flies  are  active  in.  We 
planned  further  experiments  to  replicate  this  effect  and  provide  data  for  a  more  definitive 
analysis  of  the  effect  of  the  individual  image  on  the  parameters  of  adaptation. 

2. 1.3. 5  Analysis  of Neurobiological  Data  and  Implications  for  Modeling 

Our  analysis  of  the  basic  behavior  of  biological  motion  detectors  in  response  to  the  velocity 
of  natural  scenes  was  sufficiently  complete  at  this  point  to  begin  preparation  of  several 
manuscripts  for  submission  to  specialized  vision  research  journals.  Although  our  elaborated, 
adaptive  EMD  models  provided  a  basis  for  development  of  robust  biomimetic  motion  detector 
arrays  that  are  a  clear  improvement  over  previous  implementations  of  the  Reichardt  model, 
they  have  yet  to  provide  a  definitive  explanation  for  the  remarkable  velocity  constancy 
exhibited  in  the  biological  system,  particularly  that  observed  in  (presumed)  unadapted 
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responses.  We  consider  our  discovery  of  the  full  nature  of  this  phenomenon  to  be  a  major 
finding  for  understanding  biological  motion  coding  and  prepared  a  manuscript  summarizing  the 
key  findings  for  submission  to  the  journal  Nature  (rejected  and  subsequently  modified  for 
submission  to  an  alternative  journal). 

To  summarize  the  basic  observations  of  ‘velocity  constancy’  at  this  point,  responses  of  fly 
HS  or  CH  neurons  were  found  to  be  virtually  independent  of  the  natural  image  selected,  even 
where  the  ‘natural’  contrast  of  those  images  varies  over  a  large  range.  Even  highly  elaborated 
EMD  and  tangential  cell  models  that  incorporate  many  of  the  known  static  nonlinearities  in  the 
biological  system  (including  the  HS  cell  ‘gain  control’  mechanism  proposed  by  Borst  et  al.)  fail 
to  reproduce  this  behavior  (see  Modeling).  Since  static  nonlinearities  fail  to  provide  a  perfect 
model  for  the  biological  system,  our  data  support  the  hypothesis  that  dynamic  motion 
adaptation  plays  a  role  in  the  velocity  constancy  phenomenon. 

In  addition  to  the  data  for  two  neurons  provided  in  earlier  material,  in  this  period  we 
performed  analysis  of  data  from  a  further  three,  including  one  recording  of  exceptional 
stability.  In  each  case,  these  newer  results  used  the  longer  inter-stimulus  ‘rest’  period  that  we 
concluded  was  required  following  our  previously  reported  data.  For  that  reason,  we  did  not 
pool  the  data  together  at  this  point,  and  awaited  one  or  two  further  stable  recordings  before 
doing  so. 

Figure  16  shows  the  results  of  Weibull  curve  fits  (as  outlined  earlier)  using  two  different 
images  as  the  adapting  pattern.  In  all  cases,  the  adapting  pattern  had  optimal  speed  and  was 
presented  for  4  seconds  prior  to  presentation  of  the  probe  stimulus.  One  adapting  image 
(“Hamlin  woods”)  is  the  forest  scene  with  numerous  vertical  tree  trunks  visible  against  a  bright 
background  (see  fig  3),  and  has  the  highest  contrast  of  the  set  of  images  that  we  have  available 
to  us  as  stimuli  in  our  experiments.  Several  different  contrast  measures  show  that  the  second 
image  (“Gardens”)  captured  under  dull  light  conditions  from  the  Botanic  Gardens  near  the 
University  of  Adelaide  has  1/4  to  1/3  of  the  contrast  of  the  Hamlin  woods  scene.  The 
‘canonical’  probe  stimulus  (an  optimal  frequency  sinusoidal  grating)  is  identical  in  every  case, 
except  for  its  contrast.  The  left  hand  panel  shows  the  raw  contrast  sensitivity  obtained  by 
solving  Weibull  function  fits  to  individual  contrast/response  functions  at  a  50%  maximal 
response  criterion.  Such  functions  include  both  major  components  -  gain  reduction  and  the 
‘waterfall  effect’  (an  antagonistic  DC  offset  to  membrane  voltage)  of  motion  adaptation  that  we 
identified  in  our  earlier  studies  (reported  in  Neuron,  Harris  et  al.  2000).  It  is  clear  from  the  data 
for  both  images  that  when  contrast  of  the  images  is  strongly  attenuated,  there  is  little 
observable  effect  on  contrast  threshold,  suggesting  that  both  of  these  components  are  only 
recruited  by  relatively  high  contrast  adapting  patterns.  Note  that  the  adapting  contrast  referred 
to  is  not  an  absolute  value,  but  is  expressed  relative  to  the  ‘natural’  contrast  of  the  adapting 
scene. 
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Figure  16:  Contrast  sensitivity  following  motion  adaptation  by  natural  scenes. 

At  low  contrasts  (<0.15  times  natural  contrast)  contrast  threshold  in  the  adapted  responses 
in  all  5  cases  is  close  to  that  evaluated  in  unadapted  neurons  (threshold  ca.  0.04).  When  we 
consider  that  this  threshold  is  evaluated  at  a  50%  maximal  response  criterion,  this  highlights 
the  high  inherent  gain  of  this  system.  At  relative  contrasts  for  the  adaptor  (i.e.  the  natural 
scene)  above  0.15,  the  threshold  starts  to  rise  rapidly.  All  5  curves  are  remarkably  flat  in  the 
lower  part  of  the  curve  and  are  not  distinguishable  from  unadapted  responses.  This  transition 
from  a  non-adapting  to  adapting  regime  as  pattern  contrast  increases  is  an  interesting 
observation,  suggestive  of  a  possible  ‘threshold’  in  the  adaptation  mechanism.  It  may,  however, 
also  be  consistent  with  a  shunting  inhibitory  mechanism  in  which  the  membrane  conductance 
associated  with  the  shunting  effect  falls  below  levels  associated  with  other  membrane 
conductances,  when  adaptation  is  weak. 

The  right  panel  shows  effect  of  subtracting  away  the  DC  offset  in  the  response  (the 
‘waterfall  effect’)  and  re-solving  the  criterion  response.  Although  the  thresholds  are  lower,  the 
qualitative  shapes  of  the  curves  are  similar,  suggesting  that  both  components  of  adaptation  are 
similarly  sensitive  to  contrast. 

Of  great  interest,  the  data  show  that  both  images  produce  a  shift  from  the  non-adapting  to 
adapting  regime  at  a  similar  threshold  value  (filled  versus  open  symbols),  even  though  the 
intrinsic  contrast  of  one  is  at  least  3  times  lower  than  the  other.  Furthermore,  if  we  re-plot  the 
data  from  the  most  reliable  of  the  5  recordings  obtained  so  far  (figure  17),  it  is  evident  that 
above  this  ‘onset’  contrast  for  the  gain  reduction,  the  rate  of  increase  in  threshold  (i.e.  strength 
of  adaptation)  is  different  for  the  two  images.  Hence,  we  conclude  that  the  adaptation  state  does 
depend  on  the  intrinsic  contrast  of  the  image.  It  is  worth  pointing  out  that  this  is  the  only 
experiment  we  have  yet  conducted  using  ‘true’  natural  scenes  that  has  yielded  a  clear 
difference  in  response  to  these  two  images.  Motion-adapted  velocity  tuning  curves  superimpose 
perfectly  at  any  one  ‘attenuation  level’  for  such  scenes,  so  that  the  simultaneous  presence  of 
velocity  constancy  and  differential  adaptation  has  been  demonstrated. 
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Figure  17:  Contrast  sensitivity  data  for  a  single  neuron. 

Since  the  results  of  these  experiments  strongly  support  a  role  for  motion  adaptation  in  the 
velocity  constancy  phenomenon,  and  our  models  for  static  nonlinearities  fail  to  reproduce  the 
same  behavior,  a  question  that  we  asked  at  this  stage  was,  why  do  (apparently)  unadapted 
neuron  responses  display  near-  ‘velocity  constancy’  to  the  same  set  of  images?  With  respect  to 
this  observation,  it  must  be  noted  that  our  standard  stimulus  techniques  for  evaluating  such 
responses  were  not  able  to  exclude  the  possible  contribution  of  very  rapid  components  of 
adaptation:  This  is  because  probe  stimuli  were  presented  after  subjecting  the  neuron  to  prior 
‘stimulation’  by  a  blank  screen  of  mean  luminance.  When  the  blank  screen  was  replaced  with 
the  pattern  in  motion,  response  transients  were  evoked  that  then  dissipate  with  the  time 
constant  of  any  temporal  filters  in  the  motion  pathway.  Since  we  need  to  exclude  such 
transients  from  our  analysis,  we  needed  to  discard  the  first  50-100  ms  of  data  collected.  Hence, 
if  adaptation  is  rapid  enough,  it  may  already  have  influenced  the  responses  that  we  evaluated. 

2. 1.3. 6  Alternative  stimuli  for  evaluating  unadopted  responses 

Are  any  alternative  stimulus  techniques  available  to  evaluate  responses  of  unadapted 
neurons  to  high  contrast  stimuli?  One  method  that  we  developed  previously  to  avoid  the 
confounding  effects  of  motion  adaptation  on  raw  response  tuning  is  the  ‘contrast  ramp’ 
method,  in  which  the  contrast  of  the  probe  stimulus  is  increased  over  time,  from  an  initial  zero 
value.  This  ‘soft’  onset  of  this  stimulus  avoids  the  transient  behavior  referred  to  above  and  was 
regarded  as  a  potentially  useful  alternative  stimulus  regime.  Unfortunately,  this  stimulus  is  not 
suitable  for  ‘probing’  the  contrast  sensitivity  of  the  adapted  response,  as  it  confounds  recovery 
from  adaptation  (during  the  early  part  of  the  ramp)  with  gain.  Figure  illustrates  an  interesting 
pilot  experiment  with  this  stimulus  using  a  slower  ramp  than  we  used  previously  (5  seconds 
versus  1  second).  In  each  case  the  stimulus  was  a  sinusoidal  grating  which  increased  in  contrast 
linearly  over  a  5  second  period.  An  interesting  feature  of  such  a  stimulus  is  that  as  the  ramp 
reaches  the  contrast  threshold  for  any  given  speed  of  the  pattern,  the  weakly  stimulated  neuron 
is  within  the  ‘Reichardt  detector’  part  of  its  dynamic  range  and  thus  subsequent  increase  in 
response  is  approximately  quadratic  until  either  the  effects  of  saturation  or  adaptation  start  to 
distort  its  shape  and  response  plateaus.  This  is  clearly  visible  in  most  of  the  raw  data  traces  in 
Figure,  at  least  at  lower  temporal  frequencies  (pattern  speed). 
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In  such  data,  the  contrast  sensitivity  at  any  particular  temporal  frequency  is  explicitly 
represented  by  the  time  at  which  responses  reach  threshold.  The  solid  vertical  line  in  Figure  18 
shows  that,  as  expected  from  the  temporal  frequency  tuning  characteristic  of  a  Reichardt- 
detector  based  system,  this  threshold  value  (say  at  50%  maximum  response  criterion)  is  very 
similar  at  very  low  (e.g.  0.5  Hz)  and  very  high  (40Hz)  frequencies,  while  threshold  is  reached 
earlier  (i.e.,  at  lower  contrasts)  at  intermediate  (i.e.  optimal)  frequencies.  The  most  interesting 
feature  that  is  immediately  evident  from  this  experiment  is,  however,  that  once  threshold  has 
been  reached,  response  time  course  is  very  different  over  the  subsequent  part  of  the  ramp  at 
very  high  temporal  frequencies.  In  particular,  it  never  reaches  a  ‘steady  state’  plateau  level  as 
predicted  by  static  models  for  non-linear  ‘gain  control’  (i.e.,  the  Borst  model),  but  rather  falls 
back  to  the  resting  membrane  potential  (dotted  line  on  each  trace)  and  even  below  at  very  high 
contrasts.  This  suggests  that  motion  adaptation  during  the  ramp  is  so  rapid  that  it  may 
overcome  the  quadratic  rise  in  supra-threshold  response  -  but  only  at  high  temporal 
frequencies. 
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Figure  18:  Raw  data  for  a  CH  neuron  in  response  to  5  second  contrast  ramps  of  a 
sinusoidal  grating  moving  in  the  preferred  direction  (left)  or  anti-preferred  direction 
(right)  at  a  range  of  temporal  frequencies.  Data  obtained  by  D.  O’Carroll 

Is  this  effect  due  to  the  waterfall  or  gain  reduction  component  of  adaptation?  The  simple 
test  for  this  is  to  repeat  the  experiment  in  the  anti-preferred  direction,  since  our  previous  work 
shows  that  this  does  not  recruit  a  significant  waterfall  component.  The  right  column  in  fig  18 
shows  that,  again,  the  confounding  effect  of  gain  reduction  on  time-course  is  very  evident  at 
high  temporal  frequency  but  even  visible  at  intermediate  frequencies,  where  responses  reach 
saturation,  adapt  back  below  this  level  in  the  middle  of  the  ramp  and  then  rise  again  at  the  end 
of  the  ramp  (particularly  evident  at  5-10  Hz).  Interestingly,  while  preferred  direction  responses 
at  very  high  temporal  frequency  fall  below  resting  potential,  this  is  not  seen  in  the  anti- 
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preferred  direction.  This  suggests  that  both  components  of  adaptation  strongly  shape  the  time 
course  even  during  the  stimulus  at  very  high  frequencies.  The  waterfall  effect  clearly 
contributes  to  strong  asymmetry  of  the  preferred  versus  anti-preferred  responses  at  even  very 
low  contrasts  and  within  a  very  short  time  after  onset  of  the  ramp.  This  is  a  surprise  given  that 
we  had  previously  assumed  the  waterfall  component  to  be  relatively  sluggish.  Taken  together 
with  the  data  presented  earlier  in  this  section  (fig  17),  our  latest  data  suggests  that  the 
‘waterfall’  component  of  adaptation  may  play  a  significant  role  in  the  velocity  constancy 
phenomenon. 

The  data  are  consistent  with  a  hypothesis  that  the  time-scale,  and  possibly  the  strength  of 
adaptation,  depend  on  the  temporal  structure  of  the  stimulus.  Specifically,  adaptation  may  be 
faster  (and/or)  stronger  at  higher  temporal  frequencies.  In  the  context  of  adaptation  to  natural 
images,  it  is  interesting  to  speculate  that  this  previously  unsuspected  degree  of  adaptation  to 
such  high  temporal  frequency  stimuli  might  play  a  critical  role:  Such  components  would  be 
produced  by  the  higher  spatial  frequency  features  of  natural  scenes  at  a  range  of  image  speeds. 
Given  the  basic  1/f  property  of  natural  scenes,  they  would  only  be  low  contrast  elements  of 
such  scenes,  but  fig  1 8  shows  that  such  adaptation  is  evident  at  even  low  contrasts  during  the 
earliest  parts  of  the  ramps. 

In  principle,  such  a  mechanism  could  produce  a  constant  adaptation  level  for  a  variety  of 
adapting  speeds  in  a  natural  (1/f)  image,  even  if  implemented  as  a  feedback  gain  reduction:  As 
speed  increases,  the  temporal  frequency  spectrum  passed  by  the  low-pass  optical  filtering  of  the 
eye  shifts  upward  (to  recruit  faster  and  stronger  adaptation).  Although  such  faster  patterns  may 
be  beyond  the  optimal  speed  range  of  the  neuron  (and  thus  produce  weak  global  excitation  of 
the  tangential  cell)  the  inherently  higher  contrast  of  the  low  spatial  frequencies  that  produce 
high  temporal  frequencies  in  a  fast  moving  scene  could  mitigate  this  effect  and  still  produce 
strong  local  adaptation.  Such  a  mechanism  would  only  ‘fall  out  of  compliance’  at  much  lower 
speeds,  which  would  fail  to  produce  any  adaptation  at  all.  In  this  context,  it  is  particularly 
interesting  to  note  that  the  pilot  experiments  we  conducted  recently  (reported  several  months 
ago)  showed  a  surprisingly  strong  and  yet  constant  adaptation  level  for  even  the  highest 
adapting  speeds  we  can  produce,  even  though  such  stimuli  produce  little  steady  state  response 
from  the  neuron.  These  data,  by  way  of  contrast,  showed  significant  decline  in  the  strength  of 
adaptation  to  slow  moving  patterns.  Thus  the  ‘velocity  tuning’  of  adaptation  is  not  identical  to 
the  velocity  tuning  of  the  tangential  cell  itself. 

This  idea  clearly  requires  extensive  further  physiological  work  to  provide  a  basis  for  further 
modeling.  In  the  shorter  term,  however,  it  may  be  worth  reconsidering  the  present  inclusion  of 
a  low-pass  filter  in  the  adaptive  element  of  our  existing  EMD  models:  the  latter  was  included 
on  the  assumption  that,  to  provide  a  gain  reduction  that  compensates  for  the  difference  in 
contrast  between  different  images,  local  EMDs  ought  to  adapt  to  a  level  that  reflects  the  global 
luminance  contrast  of  the  image  (a  parameter  that  can  be  extracted  from  the  signal  by  a 
combination  of  partial  or  full  rectification  with  low-pass  filtering,  as  in  our  models).  It  will  be 
interesting  to  explore  the  effect  of  replacing  this  low  pass-filter  with  a  temporal  band-pass 
mechanism  that  would  reject  the  lower  frequency  components  of  such  scenes  and  introduce  a 
dependence  on  temporal  frequency  into  the  adaptation  mechanism.  During  our  earlier  work  for 
the  concurrent  SBIR  Phase  I  effort,  we  already  showed  (using  Fourier  analysis  of  individual 
EMD  outputs  in  a  relatively  unelaborated  EMD  model)  that  intermediate  temporal  frequencies 
in  the  output  of  opponent  EMDs  were  inherently  independent  of  the  speed  at  which  the  pattern 
was  moved  and  thus  might  provide  an  alternative  to  seeking  the  DC  response  through  massive 
low-pass  filtering  in  the  gain  control  mechanisms.  Using  a  bandpass  filter  tuned  to  higher 
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temporal  frequencies  (10-50  Hz)  as  the  source  of  either  the  gating  signal  to  enable  feedforward 
contrast  normalization  or  a  feedback  gain  control  would  also  yield  a  much  shorter  time- 
constant  for  adaptation,  at  least  at  higher  temporal  frequencies. 

By  the  onset  of  winter,  we  had  obtained  contrast  sensitivity  data  from  a  number  of  further 
neurons,  including  4  HS  cells,  1  CH  cell,  and  1  VS  cell.  These  data  included  tests  in  which  both 
velocity  and  contrast  of  the  adapting  stimuli  were  varied  as  parameters.  Reduction  and  analysis 
of  the  new  data  continued  through  the  winter,  in  preparation  for  presentations  by  the  PI  and  T. 
Rainsford  at  two  conferences  in  Brisbane  (August  2004,  see  below).  During  this  period,  T. 
Rainsford  accepted  a  new  position  in  another  laboratory.  Experimental  work  during  the 
remaining  part  of  the  reporting  period  were  obtained  by  a  new  post-doctoral  researcher  in  the 
O’Carroll  lab,  Karin  Nordstrom,  who  was  supported  in  by  by  AFOSR  funds,  but  also  by  the 
Swedish  Research  Council.  Subsequent  experiments  were  also  carried  out  with  AOARD 
funding  by  a  part-time  technical  assistant,  Irene  Moyer,  through  to  the  end  of  the  period.  The 
training  of  these  new  personnel  necessarily  introduced  some  delay  into  the  analysis  and 
execution  of  further  experiments. 

From  mid-winter  (June  2004)  in  Adelaide,  flies  were  not  easily  obtained.  Although  some 
additional  recordings  were  performed  at  the  start  of  the  period,  they  were  not  sufficient  to 
address  the  major  aim  of  studying  further  the  effect  of  adapting  conditions.  Instead  we  worked 
up  the  backlog  of  data  from  recent  experimental  recording  periods  into  figures  with  high 
enough  quality  for  use  in  publications  and  talks  at  the  22nd  Quadrennial  meeting  of  the 
International  Congress  of  Entomology  (ICE)  and  subsequent  workshop  “Insect  Sensors  & 
Robotics”  (ISR)  at  the  University  of  Queensland. 

We  expected  flies  to  be  more  readily  available  during  the  upcoming  reporting  periods  and 
planned  to  continue  HS  recordings  using  the  test-adapt-test  approach  with  the  intention  of 
estimating  the  time  constant  of  adaptation.  Given  some  contemporary  simulation  results  (ijn 
our  concurrent  Air  Force  funded  SBIR  Phase  II  project),  which  suggested  that  an  adaptive  time 
constant  of  200ms  or  even  faster  may  be  optimal  for  controlling  contrast  gain,  we  considered  it 
a  priority  in  further  recordings  to  vary  the  adapting  period,  (using  both  optimal  and  non- 
optimal  adapting  patterns)  in  addition  to  the  recent  efforts  varying  adapting  contrast  and  speed. 

While  little  progress  was  made  on  actual  experiments  due  to  poor  availability  of  Eristalis 
this  spring,  we  did  considerable  work  in  upgrading  laboratory  facilities.  Interface  software  was 
added  to  the  Vision  Egg  by  an  Honours  student,  Lachlan  Dowd,  which  greatly  facilitates  the 
design  and  running  of  complex  experimental  sequences,  and  which  we  expected  to  increase  the 
volume  and  range  of  data  available  from  individual  experiments.  In  addition,  we  furnished  a 
second  experimental  bench  in  the  lab,  enabling  parallel  experiments  to  be  run. 

2.1.4  Year  4  (2005):  end  of  experimental  period 

An  unusually  late  Spring  season  meant  that  little  progress  was  made  on  experiments  due  to 
poor  availability  of  Eristalis.  As  flies  became  available,  we  began  training  our  research 
assistant  Irene  Moyer,  who  started  work  part-time  with  the  intention  of  recording  from  HS  cells 
when  Eristalis  became  more  abundant. 

2. 1.4.1  Strip  test  experiments  to  evaluate  waterfall  effect 

A  hitherto  poorly  studied  aspect  of  the  neurons  response  was  the  waterfall  effect. 
Successful  recordings  were  made  in  this  period  by  both  Irene  Moyer  and  post-doctoral  fellow 
Karin  Nordstrom  and  Irene  Moyer.  Experiments  aimed  to  address  2  key  questions  relating  to 
the  poorly  characterized  waterfall  effect.  The  first  set  of  experiments  (“Strip-test,  surround 
adapt”)  used  a  sinusoidal  stimulus  in  a  probe-adapt-probe  experiment  to  evaluate  gain  change 


n  a  A/m  D’riBRnTi  T  Tmtvpr  sttv  of  A  fipt  a  mu 


AOARD-03-4026  /  RMO  No  2001/161 1 


43 


resulting  from  adaptation,  as  in  the  previous  experiments  on  the  effect  of  adapting  speed  and 
contrast.  As  before,  data  were  analysed  using  a  Weibull  function  fit  to  the  contrast  sensitivity 
function.  In  each  case,  the  stimulus  was  the  same  ‘canonical’  sinusoidal  grating  (with 
approximately  optimal  spatial  and  temporal  frequency)  used  as  a  probe  stimulus  in  the  earlier 
experiments.  The  difference  compared  with  earlier  applications  of  similar  stimuli  was  that  the 
grating  was  used  as  the  adapting  stimulus,  as  well  as  the  probe,  but  probed  different  parts  of  the 
receptive  field  of  the  HS  neuron  to  that  adapted.  This  was  achieved  by  using  a  grating  with  a 
central  ‘strip’  missing  as  the  adapting  stimulus,  while  probing  the  central  part  of  the  receptive 
field  with  the  missing  strip  alone.  The  rationale  behind  this  approach  is  that  we  know  from 
earlier  work  (e.g.  Maddess  &  Laughlin  1986)  that  the  main  effect  of  adaptation  (quantified  by 
ourselves  as  a  contrast  gain  reduction)  is  a  highly  local  phenomenon.  It  is  unclear  whether  the 
waterfall  effect  is  also  locally  generated,  but  because  it  is  a  persistent  effect,  it  can  potentially 
influence  the  membrane  potential  of  the  HS  cell  even  when  stimuli  are  presented  to  previously 
parts  of  the  receptive  field.  Hence,  the  ‘surround’  adaptation  allows  us  to  probe  the  effect  of  the 
waterfall  component  on  ‘normal’  responses  to  a  part  of  the  visual  field  that  is  unaffected  by 
adaptation.  Furthermore,  because  our  earlier  work  (Harris  et  al  2000)  had  shown  that  the 
waterfall  effect  is  direction  selective,  we  can  control  in  these  experiments  by  using  different 
combinations  of  adapt  and  test  stimuli  in  the  preferred  and  anti-preferred  (null)  direction.  A 
clear  prediction  based  on  our  earlier  work  would  be  that  null-direction  adaptation  of  the 
surround  would  recruit  no  waterfall  effect  and  thus  affect  neither  contrast  gain,  nor  global 
membrane  potential.  We  would  thus  expect  to  observe  no  effect  of  such  adaptation  on  contrast 
sensitivity  functions  in  either  preferred  or  null  directions.  By  contrast,  any  ‘global’  contribution 
of  the  waterfall  effect  accumulating  during  preferred  direction  adaptation  would  cause  an 
effective  DC  (downward)  offset  in  the  contrast  sensitivity  functions  (for  either  stimulus 
direction),  and  thus  we  can  quantify  their  contribution  to  gain  reduction. 

The  basic  stimulus  and  examples  of  responses  in  raw  trials  are  shown  below,  with  data 
illustrated  for  3  contrasts  (1,0.1  and  the  control  of  0).  In  each  case,  data  are  illustrated  for  (a) 
null  direction  test  and  adapt  (b)  null  direction  test,  but  preferred  adapt  (c)  preferred  direction 
test,  null  adapt  and  (d)  preferred  direction  test  and  adapt.  The  test  periods  are  indicated  by  gray 
boxes,  adapt  by  the  black  stimulus  bar.  It  is  immediately  evident  from  the  raw  data  that  effects 
on  the  second  test  stimulus  are  rather  subtle,  even  at  the  low  contrast  illustrated.  This  is 
especially  noticeable  compared  with  previous  test-adapt-test  experiments  that  we  conducted 
(e.g.  Harris  et  al  2000).  Strong  adaptation  is  evident  in  both  test  and  adapt  stimuli  during  the 
time  course. 
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Figure  19:  Raw  data  for  an  HS  neuron  in  test-adapt-test  experiments  using  a  surround 
adapting  stimulus  and  strip  test.  Data  are  shown  for  3  test  contrasts  (test  periods  indicated 
by  gray  bars)  and  4  combinations  of  adapt  and  test  direction. 

Figure  20  below  illustrates  results  for  a  single  HSE  neuron  in  Eristalis.  This  data  illustrates 
the  main  findings  of  the  overall  set  of  data  recorded  from  1 8  HS  neurons  to  date.  In  each  case, 
there  is  little  or  no  evidence  for  any  right  shift  in  adapted  contrast  sensitivity  functions, 
confirming  earlier  findings  that  contrast  gain  reduction  is  a  local  phenomenon. 
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Figure  20:  Contrast  sensitivity  data  for  a  single  HS  neuron  recorded  in  test-adapt-test 
experiments  using  a  surround  adapting  stimulus  and  strip  test.  Data  are  shown  for  4 
combinations  of  adapt  and  test  in  preferred  (P)  and  null  (N)  directions.  Individual  data 
points  are  shown  together  with  means  from  all  trials  at  each  contrast.  The  dotted  lines 
represent  the  same  data  after  subtracting  away  the  response  at  contrast=0  (the  control 
experiments  shown  in  the  right  hand  side  of  the  previous  figure).  Data  recorded  by  Karin 
Nordstrom,  Irene  Moyer. 


Figure  21  below  shows  similar  data  averaged  across  6  FISE/F1SNE  neurons,  after 
normalizing  each  curve  to  take  account  of  differences  in  maximum  response  (due  to  differences 
in  penetration  quality,  as  outlined  in  the  earlier  reporting  periods).  Perhaps  the  biggest  surprise 
from  these  experiments  is  that  the  surround  adaptation  reveals  antagonistic  afterpotentials  een 
when  adapting  stimuli  are  presented  in  the  null  direction.  This  partially  contradicts  our  earlier 
findings  (Flarris  et  al  2000).  Note  that  the  very  ‘early’  response  period  used  for  this  analysis 
(between  100  and  300  ms  after  stimulus  onset)  may  pick  up  a  brief,  transient  ‘burst’  of 
spikelets  in  some  recordings  upon  relaxation  from  the  hyperpolarizing  adapting  stimulus:  such 
transients  are  clearly  evident  in  the  raw  data  illustrated  earlier. 
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Figure  21:  Contrast  sensitivity  data  (as  in  previous  figure)  but  averaged  across  6  neurons. 
Error  bars  denote  s.e.m.  Data  are  shown  for  4  combinations  of  adapt  and  test  in  preferred 
(P)  and  null  (N)  directions.  Individual  data  points  are  shown  together  with  means  from  all 
trials  at  each  contrast.  The  dotted  lines  represent  the  same  data  after  subtracting  away  the 
response  at  contrast=0  (the  control  experiments  shown  in  the  right  hand  side  of  the 
previous  figure).  Data  recorded  by  Karin  Nordstrom,  Irene  Moyer. 


To  summarize  the  results  of  these  experiments,  the  recordings  show  that  the  waterfall 
effect,  when  present,  adds  a  DC  offset  to  the  response  only  when  test  stimuli  are  presented  in 
the  same  direction  as  the  adapting  stimuli  (PPP,  NNN).  In  this  case,  the  offset  persists  even  at 
the  highest  test  contrasts  (i.e.  even  when  the  underlying  responses  are  fully  saturated).  This 
suggests  that  the  waterfall  effect  is  independent  of  the  site  of  response  saturation  in  the  motion 
pathway.  When  test  stimuli  are  different  in  direction  to  the  adaptor  (NPN,  PNP),  a  DC  offset  is 
still  evident  at  very  low  contrast,  but  responses  at  higher  contrasts  saturate  at  the  same  level  as 
before  adaptation.  This  makes  sense  since  in  this  case  the  waterfall  effect  offset  is  antagonistic 
to  the  adaptor  and  thus  adds  to  the  test  response  measured.  At  very  low  contrasts,  there  is  no 
response  to  the  test  stimulus  itself,  so  the  waterfall  effect  is  the  only  measurable  voltage. 

The  final  set  of  experiments  used  a  simpler  experimental  approach  of  a  long  stimulus  (4 
seconds  duration)  applied  to  an  unadapted  neuron,  using  the  3  images  (‘Hamlin’,  ‘Close’  and 
‘Gardens’)  that  represent  the  extremes  of  contrast  in  our  set  of  natural  panoramas,  as  well  as  for 
our  canonical  sinusoidal  stimulus.  Response  was  simply  evaluated  in  response  windows  very 
early  in  the  stimulus  ( Initial  in  figure  22  below)  at  the  end  of  4  seconds  ( Adapted )  and  the 
antagonistic  response  immediately  after  stimulus  end  ( waterfall ) 
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Figure  22:  Averaged  responses  of  an  HS  neuron  to  4  seconds  of  stimulation  with 
panoramic  natural  scenes,  illustrating  the  3  periods  (indicated  in  gray)  used  to  evaluate 
responses.  Data  recorded  by  Karin  Nordstrom. 

Figure  23  below  shows  the  response  averaged  across  these  3  windows  when  the  speed  of 
the  pattern  is  varied,  for  each  of  the  3  images  and  also  for  the  sinusoidal  image.  The  data  are 
the  mean  of  3  HS  neurons,  each  normalized  to  account  for  responsiveness  (as  before).  Both 
initial  and  adapted  responses  show  clear  velocity  constancy,  as  expected  from  our  earlier 
experiments.  Adapted  responses,  however,  are  substantially  weaker  due  to  the  combined  effects 
of  both  gain  reduction  and  the  waterfall  component.  Interestingly,  the  shape  of  the  responses 
are  similar  in  each  case  if  they  are  re-normalized  to  their  own  maximum  (bottom  right). 

The  main  point  of  this  experiment  is  illustrated  by  the  top  right  panel:  It  can  be  seen  that  the 
waterfall  effect  is  largest  at  the  same  velocities  that  the  underlying  response  is  strong.  Again,  if 
we  invert  this  curve  and  normalize  it  to  its  own  maximum,  the  resultant  function  nearly  overlies 
the  basic  velocity  tuning  of  the  HS  cell.  In  this  respect  the  waterfall  effect  is  clearly  distinct 
from  the  gain  reduction  that  we  quantified  in  the  earlier  reporting  periods,  which  showed 
somewhat  more  tuning  to  higher  image  velocities.  Taken  together  with  the  results  above,  we 
conclude  that  the  waterfall  effect  is  clearly  a  separate  component  to  the  gain  reduction  and  is 
most  likely  a  property  of  the  HS  neuron  itself,  probably  being  generated  within  the  axon,  since 
the  offset  it  introduces  does  not  prevent  underlying  responses  from  saturating. 
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Figure  23:  Averaged  responses  of  3  HS  neuron  to  4  seconds  of  stimulation  with  panoramic 
natural  scenes,  evaluated  in  the  response  periods  indicated  in  the  previous  figure.  The 
bottom  right  panel  shows  the  data  normalized  by  dividing  through  by  their  maximum. 
Data  recorded  by  Karin  Nordstrom. 


2. 1.4. 2  Duration  of  adapting  stimuli 

Perhaps  the  most  intriguing  aspect  of  the  project  to  this  point  was  the  observation  that  the 
(apparently)  unadapted  responses  of  neurons  presented  with  natural  images  already  display 
velocity  constancy  within  the  first  few  hundred  milliseconds  of  the  experiment  (figs  6-8).  This 
property  is  summarized  for  all  of  the  neurons  studied  so  far  in  figures  24  and  25  below,  taken 
from  our  submitted  manuscript  (Straw,  Rainsford  &  O’ Carroll,  presently  under  consideration 
by  the  journal  Current  Biology). 
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Figure  24:  Unadapted  responses  in  an  HS  neuron  to  1  second  of  stimulation  with 
panoramic  natural  scenes.  A  shows  the  experimental  system.  B,C  responses  to  unsealed 
and  contrast  reduced  versions  of  images  A,D,H  (see  figure  3)  (from  Straw,  Rainsford  & 
O’Carroll  2005,  submitted). 
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Figure  25:  Velocity  tuning  (showing  velocity  constancy)  in  unadapted  and  motion  adapted 
responses  of  HS  neurons  to  both  preferred  and  anti-preferred  directions  of  motion.  A 
shows  the  unadapted  responses  to  scaled  (left)  and  two  degrees  of  artificial  contrast 
compression  (right).  B  shows  similar  data  for  neurons  adapted  using  the  the  monotonic 
stimulus  sequence  described  earlier  (and  see  also  figure  26)  (from  Straw,  Rainsford  & 
O’Carroll  2005,  submitted). 

These  data  make  a  very  clear  case  for  velocity  constancy,  but  yet  are  enigmatic  because  the 
artificially  contrast-reduced  experiments  are  always  consistent  with  one  of  the  basic  predictions 
of  the  Reichardt  model  -  that  lower  contrast  stimuli  should  produce  lower  contrast  responses. 
Intriguingly,  at  no  point  do  the  reduced  contrast  responses  fall  out  of  compliance  with  respect 
to  one  another.  This  argues  strongly  against  the  velocity  constancy  being  due  solely  to 
saturation  nonlinearities. 

An  interesting  insight  into  this  problem  conies  from  examination  of  our  ‘population’  data 
for  large  numbers  of  repetitions  of  experiments  in  the  adapted  state,  using  the  monotonic 
method  (figure  26).  Because  this  method  is  quick  to  generate  results,  we  are  able  to  evaluate  a 
larger  set  of  images.  What  is  most  interesting  is  that  the  artificial  images  -  those  taken  from 
man-made  scenes  (images  E  and  I,  shown  with  open  symbols  to  distinguish  them  from  the 
natural  scene  panoramas)  are  outliers  in  the  contrast  reduced  set,  with  obviously  larger 
responses  at  all  speeds.  This  is  less  evident  but  perhaps  still  present  in  the  unsealed  images. 
This  observation  is  significant  because  these  images  are  by  no  means  the  highest  global 
contrast  of  the  set.  Indeed,  image  I  is  the  lowest,  with  1/4  the  global  contrast  of  image  A  by 
several  measures.  This  trend  is  thus  the  complete  opposite  of  what  one  would  expect  from  a 
Reichardt  based  EMD.  Indeed,  we  confirmed  this  in  our  recently  submitted  paper  by  using  a 
contrast  measure  that  was  based  on  a  simple  EMD  model. 
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Figure  26:  Velocity  tuning  (showing  velocity  constancy)  in  motion  adapted  responses  of  HS 
neurons  to  preferred  direction  of  motion.  A  shows  the  monotonic  stimulus  sequence  from 
which  velocity  tuning  was  obtained  (as  described  earlier),  (from  Straw,  Rainsford  & 
O’Carroll  2005,  submitted). 


This  observation  is  suggestive  that  dynamic  motion  adaptation  does  play  a  role  in  velocity 
constancy,  yet  seems  to  contradict  the  observation  that  supposedly  unadapted  neurons  (fig  24, 
25)  already  display  similar  velocity  constancy.  This  interpretation  is  confounded  by  the 
difficulty  of  carrying  out  experiments  that  evaluate  a  ‘true’  unadapted  neuron  since  we  must 
expose  the  neuron  to  motion  (which  could  induce  rapid  adaptation)  in  order  to  evaluate  its 
response.  An  interesting  observation  in  even  the  raw  data  (figure  24)  is  that  little  adaptation  is 
evident  during  the  1  second  stimulus  presentation  in  these  experiments,  either  in  normal 
contrast  (b)  or  reduced  contrast  (c)  images.  This  raises  the  interesting  possibilities  that 
adaptation  is  so  fast  with  these  natural  scenes  that  the  gain  reduction  component  of  adaptation 
is  already  in  ‘steady  state’  within  the  initial  response  period.  The  slower  reduction  in  response 
observed  in  many  earlier  studies  may  be  less  associated  with  the  gain  reduction  component  of 
adaptation  than  with  the  waterfall  effect. 

The  rapid  adaptation  evident  in  the  contrast  ramp  responses  illustrated  in  fig  18  is 
consistent  with  this  hypothesis,  but  does  not  permit  is  to  evaluate  the  speed  of  adaptation 
quantitatively.  To  test  the  hypothesis  that  there  may  be  ‘fast’  components  of  motion  adaptation 
more  directly,  we  therefore  returned  to  the  probe-adapt-probe  method  and  contrast  sensitivity 
(=gain)  estimation  method  used  to  evaluate  the  effect  of  adapting  contrast  and  adapting  pattern 
speed.  In  this  case,  however,  we  varied  the  duration  of  the  adapting  stimulus  in  the  range  from 
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0.05  to  5  seconds.  At  the  time  of  writing,  we  have  obtained  several  pilot  experimental  data  sets 
with  this  stimulus,  but  have  (as  yet)  evaluated  only  one  of  these  in  detail.  The  raw  results  at 
several  different  adaptation  times  are  illustrated  in  figure  27  below. 
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Figure  27:  Contrast  sensitivity  of  an  HS  neuron  before  (black)  and  after  (red)  different 
durations  of  adaptation  in  test-adapt-test  experiments.  The  adapting  duration  is  shown  in 
the  captions.  Data  from  Irene  Moyer  &  D.  O’Carroll. 

The  right  shift  in  the  curves  shows  the  amount  of  contrast  gain  reduction  induced  by  the 
adapting  stimulus.  Two  surprising  observations  emerge:  (1)  even  by  the  longest  adapting 
stimulus  used  in  this  pilot  study,  the  right  shift  is  still  continuing  to  enlarge,  suggesting  (as 
hinted  at  in  an  earlier  experiment)  contrast  gain  reduction  continues  for  many  seconds  after 
stimulus  onset.  (2)  Even  very  short  adapting  stimuli  induce  a  measurable  gain  reduction.  Much 
further  work  is  required  to  elaborate  this  finding,  but  it  is  clear  that  contrast  gain  reduction 
begins  very  rapidly  but  continues  slowly  over  many  seconds. 

The  results  are  summarized  by  figure  28  below,  which  shows  contrast  sensitivity 
(1 /contrast  threshold  @  50%  maximal  response)  extracted  from  Weibull  function  fits  to  the 
data  in  fig  27.  The  blue  line  shows  the  adapted  response  after  allowing  for  the  confounding 
effect  of  the  waterfall  component  (fitted  by  the  offset  term  in  the  Weibull  curves  see  fig  12  and 
associated  discussion).  Unadapted  and  adapted  curves  only  converge  for  the  shortest  adapting 
stimulus  (0.05  seconds).  Importantly,  this  latter  value  is  close  to  the  estimated  delay  time 
constant  of  the  underlying  motion  detectors.  This  suggests  that  adaptation  begins  on  the  same 
time  course  as  the  response  of  the  EMDs  themselves.  If  so,  this  confounds  our  attempts  to 
quantify  any  ‘truly’  unadapted  responses,  and  we  cannot  rule  out  this  mechanism  playing  a  key 
role  in  velocity  constancy  observed  in  our  earlier  work  (e.g.  figures  24-26). 
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Figure  28:  Contrast  sensitivity  of  an  HS  neuron  before  (black)  and  after  (red)  different 
durations  of  adaptation  in  test-adapt-test  experiments.  The  adapting  duration  is  shown  in 
the  captions.  Data  from  Irene  Moyer  &  D.  O’Carroll. 


One  obvious  deficiency  of  this  experimental  design,  at  least  for  the  long  adapting  stimuli,  is 
that  we  are  not  allowing  sufficient  time  for  recovery  from  adaptation  in  one  repetition  of  this 
stimulus  before  hitting  the  neuron  with  the  next:  This  is  evident  from  the  downward  trend  in 
contrast  sensitivity  of  the  ‘unadapted’  response,  which  signifies  reduced  gain  in  these  cases. 
We  clearly  need  to  do  further  experiments  with  yet  longer  adapting  stimuli  still  as  well  as 
longer  ‘rest’  periods  between  trials  to  allow  neurons  to  fully  recover. 


2.2  Simulation  and  Modeling 

2.2.1  Year  1 

2. 2. 1.1  1 -dimensional  EMD  model  with  adaptive  (feedback)  gain  control 

Work  began  on  development  of  an  EMD  array  simulation  based  on  an  elaborated 
Reichardt  model  as  used  in  our  previous  work  (Dror  et  al  2000).  This  was  implemented  in  the 
MATLAB  environment.  This  EMD  model  includes  specific  features  designed  to  mimic  the 
obvious  known  features  of  the  neurobiology,  e.g.,  with  prefiltering  based  on  models  for 
photoreceptor  and  laminar  temporal  responses.  The  array  was  initially  one-dimensional  but  was 
intended  for  expansion  to  two  dimensions  in  the  future.  Adaptation  was  not  initially  enabled, 
but  would  be  included  as  experimental  work  was  done,  and  data  to  refine  our  models  of 
adaptation  became  available.  The  basic  framework  employed  for  development  of  this  adaptive 
EMD  is  illustrated  below  in  figure  29: 
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(1)  Whole  image  pre-processing 
Contrast  (multiplication) 

Spatial  filtering  ->  convolution  with  2-d  gaussian  kernel  (Ap) 
Normalization  (zero  mean,  logarithmic  compression) 

(2)  Velocity  dependent  temporal  pre-filtering  (single  row) 

Temporal  filtering  ->  convolution  with  Log-normal  kernel 
(parameters  Tp,  o) 

Sub-unit  indexing  based  on  spatial  separation,  A ty,(S1,  S2) 


(3)  Delay,  multiply,  subtract  (for  loop) 


Temporal  delay  filtering  (Sid,  s2d)  ->  1st  order  Butterworth  low-pass  (short  x) 

Sub-unit  multiplication  (St  x  Sd2  X  feedback  gain)  - 

Compressive  non-linearity  (tanh) 

-|  Sub-unit  subtraction  (SI,  S2) 

Full-wave  rectification 

Temporal  filtering  ->  1st  order  Butterworth  low-pass  (long  x)  | 

Figure  29:  Summary  of  3  key  components  of  elaborated  EMD  model  developed  in  Matlab.  D. 
O’Carroll. 

The  model  begins  with  the  panoramic  imagery  used  in  the  neurobiology  experiments. 
Although  we  desire  a  fully  ‘biomimetic’  front  end  for  such  a  model,  this  isn’t  practical  if  we 
are  to  use  the  same  stimuli  in  both  model  and  experiments,  because  the  camera  used  to  capture 
the  images  necessarily  imposes  some  limitations  on  the  data  obtained.  For  example,  in  nature, 
each  and  every  photoreceptor  (or  second  order  neuron)  adapts  locally  to  the  luminance  so  that 
image  sampling  cannot  be  defined  by  a  single  global  gain  terms,  whereas  in  our  camera,  a 
single  gain  governs  the  value  sampled  by  every  pixel  (within  the  limited  dynamic  range  of  the 
sensor).  Hence,  in  setting  the  camera  to  ‘optimal’  exposure  for  the  sampled  scene,  we  are 
already  biasing  the  subsequent  model  by  the  inherently  linear  sampling  strategy  of  a  CCD 
device.  In  converting  the  image  (grayscale)  to  input  values  for  the  model  ,  we  need  to  make 
assumptions  about  the  way  that  luminance  is  coded.  The  first  stage  of  the  model  “Whole  image 
pre-processing”  is  thus  an  attempt  to  re-encode  the  image  into  a  biologically  plausible  form.  In 
the  initial  model,  this  involved  adjustment  of  the  contrast  (if  desired  to  mimic  the  reduction  in 
contrast  of  the  images  as  in  our  experiments),  spatial  pre-filtering  (specifically  low-pass 
filtering  to  mimic  spatial  blur  in  insect  vision)  and  image  normalization  to  a  zero  mean  and 
with  a  logarithmic  compression  to  mimic  the  logarithmic  encoding  of  luminance  by 
photoreceptors. 

In  the  second  stage,  the  model  takes  account  of  the  temporal  filtering  properties  of  the 
insect  photoreceptors,  by  animating  the  image  one  row  at  a  time  (in  a  manner  to  mimic  the 
desired  experiment)  and  then  introducing  temporal  blur  by  convolution  with  a  lognormal 
impulse  response  derived  from  experimental  data  for  insect  photoreceptors  (Howard  et  al). 
Indexing  of  the  output  of  this  operation  then  provided  the  inputs  to  the  two  arms  of  the 
elaborated  EMD  (S  in  figure  30  below).  The  index  onto  the  animated  luminance  sequence  for 
each  image  row  allowed  the  inter-receptor  angle  (delta  phi  in  fig  30  below)  to  be  matched 
appropriately  to  the  amount  of  spatial  blur  introduced  into  the  image.  For  all  simulations,  these 
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were  set  to  values  based  on  ‘typical’  fly  photoreceptors,  with  delta  phi  of  1.5  degrees,  while  the 
spatial  blur  was  fixed  with  a  Gaussian  blur  of  half- width  2  degrees.  All  models  were  carried  out 
using  full  resolution  panoramic  images,  with  approximately  8200  pixels  wide  (thus  approx.  23 
pixels/degree).  The  model  thus  provided  an  array  of  240  EMDs  to  cover  the  full  360  degrees  of 
each  image.  Animating  the  whole  array  simultaneously  avoided  simulations  suffering  from  any 
confounding  effect  of  pattern  noise,  as  the  same  features  were  always  present  at  all  times  in  the 
field  of  view  of  the  (assumed)  circular  EMD  array. 


A<- 
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Figure  30:  Block  diagram  for  an  elaborated  1-dimensional  EMD  array  model 

Following  ‘sampling’  of  the  animated  pattern,  the  delayed  (D  in  fig  30)  and  undelayed 
EMD  arms  were  implemented  by  low-pass  filters  applied  inside  a  FOR  loop  in  Matlab  for  each 
modeled  time  step.  Although  the  limitations  of  our  digital  simulation  required  discrete  time 
sampling,  we  used  a  high  sample  rate  (500  samples  per  second  minimum)  compared  with  any 
of  the  time  constants  of  the  simulated  elements,  to  approximate  a  continuous  time  series.  The 
for  loop  allowed  for  local  EMD  outputs  to  be  fed-back  to  control  the  input  gain  (see  fig  29). 
Local  outputs  were  multiplied  and  subtracted  to  produce  the  opponent  EMD  output.  In  the 
initial  adaptive  model,  outputs  were  then  rectified  and  fed  back  to  control  the  input  gain.  At 
each  time  step,  the  output  of  opponent  EMDs  was  then  taken  as  the  overall  system  output, 
either  independently  or  after  summation  across  all  240  EMDs  in  the  row. 

Figure  31  below  shows  the  output  of  100  EMDs  in  this  model  for  several  different  image 
speeds,  and  both  with  (red)  and  without  the  adaptive  feedback  turned  on  in  the  model.  The 
trace  shows  the  typical  ‘pattern  noise’  of  a  Reichardt  based  EMD  system,  despite  the  large 
number  of  local  units  averaged  -  large  fluctuations  in  output,  but  the  basic  velocity  tuning  is 
also  evident,  with  a  response  maximum  seen  at  ca.  200  degrees/second. 
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Figure  31:  Output  of  the  basic  1-dimensional  adaptive  feedback  model,  averaged  across  100  EMDs 

2.2. 1.2  2-dimensional  elaboration  of  the  adaptive  feedback  model 

We  subsequently  developed  a  2-dimensional  EMD  model  into  a  large  retinotopic  array  of 
elaborated  EMDs.  Using  an  inter-receptor  spacing  of  1.5  degrees  as  before  (an  average  value 
for  fly  optics),  we  were  able  to  simulate  a  retinotopic  EMD  array  240  wide  by  36  high.  The 
vertical  number  was  limited  by  the  angular  extent  of  our  panoramic  images  which  are  limited 
to  57  degrees  by  the  field  of  view  of  the  original  camera  used  to  capture  them.  A  major 
advantage  of  the  approach  adopted  is  that  the  input  data  are  assumed  to  be  panoramic  images, 
oriented  across  the  equator  of  the  yaw  plane,  so  that  the  array  views  360  degrees  of  space 
horizontally  (and  54  degrees  or  the  overall  57  degrees  of  the  images  vertically).  This  ‘receptive 
field’  is  similar  to  that  achieved  by  the  combined  receptive  fields  of  the  two  HSE  neurons  in 
the  (left  and  right)  lobula  plate  of  the  fly.  By  averaging  EMD  outputs  across  the  full  360-degree 
horizon,  pattern  noise  (due  to  local  differences  in  luminance  of  form)  such  as  that  evident  in 
figure  31  was  virtually  eliminated  in  the  output,  greatly  facilitating  the  use  of  the  model  to 
measure  ‘tuning’  characteristics  of  the  system  (e.g.  velocity  tuning)  for  comparison  with  that  of 
the  biological  system. 

One  improvement  on  the  1-d  modeling  approach  was  a  method  developed  for  handling  of 
the  original  image  data.  Computation  time  in  execution  of  the  earlier  model  trades  off  against 
the  resolution  with  which  simulations  are  run,  in  both  space  and  time.  Because  the  simulations 
are  discrete  in  both  domains,  while  the  biological  system  is  discrete  only  in  the  space  domain, 
some  effort  was  put  into  determining  the  detrimental  effects  of  using  low  sample  rates, 
combined  with  appropriate  anti-aliasing  techniques  to  improve  efficiency.  Although  we  had 
acquired  a  large  set  of  panoramic  images  for  the  biological  experiments  at  very  high  spatial 
resolution  our  initial  models  required  the  use  of  downscaled  images  (to  1024  pixels  or  less)  to 
simulate  responses  in  an  acceptable  time  frame.  To  achieve  acceptable  efficiency,  we  found  it 
necessary  to  use  a  variable  sample  rate  depending  on  the  velocity  of  the  pattern,  even  after 
optimized  compilation  of  key  limiting  stages.  While  this  worked,  and  allowed  us  to  verify  the 
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effectiveness  of  the  contrast-adaptation  algorithm  to  a  limited  degree,  it  limited  simulations  to  a 
single  constant  velocity  stimulus.  In  order  to  simulate  the  experiments  carried  out  on  the 
biological  system,  and  in  particular  the  “monotonic”  series  of  experiments  for  testing  motion 
adaptation  and  velocity  constancy  (where  the  neuron  is  tested  with  short  bursts  of  ‘test’  motion 
at  variable  speed  against  a  background  of  an  adapting  speed  -  see  figure  1),  we  required  an 
approach  that  utilizes  a  constant  temporal  sample  rate. 

The  approach  developed  permits  the  use  of  the  original  image  at  its  full  resolution,  avoiding 
any  statistical  artifacts  introduced  by  interpolation.  By  using  the  very  high  spatial  sample  rate 
of  this  image,  we  were  able  to  generate  a  realistic  simulation  of  even  very  slow  motion  (such  as 
during  hovering)  at  a  constant  (and  acceptably  high)  temporal  sample  rate,  without  the  time- 
consuming  interpolation  methods  required  in  the  earlier  model.  This  latter  point  was  was 
important  because  the  neurobiological  experiments  harnessed  the  real-time  interpolation  power 
of  the  Open  GL  graphics  card,  and  thus  was  able  to  provide  responses  to  very  low  stimulus 
speeds.  The  general  organization  of  this  revised  model  is  as  follows: 

1 .  Spatial  Prefiltering:  The  panoramic  image  is  spatially  pre-filtered  by  2-dimensional 
convolution  with  Gaussian  kernels  to  mimic  insect  optics  (low  pass)  and  lateral 
spatial  interactions  (lateral  inhibition,  high-pass)  by  lamina  monopolar  cells 
(LMCs).  These  filters  are  applied  sequentially  to  the  green  channel  of  the  original 
RGB  image  in  integer  form,  with  unity  gain,  outside  Matlab,  using  a  difference  of 
Gaussian  filter  applied  in  Adobe  Photoshop.  In  practise,  the  DOG  filter  is  not 
applied  in  one  pass,  but  rather  we  construct  two  Gaussian  filtered  versions  of  the 
image,  1  with  2  degree  half-width  and  one  at  6  degrees.  This  gives  the  Matlab 
model  flexibility  to  use  either  low  pass  or  band  pass  spatial  filters  by  applying  the 
difference  operation,  as  desired. 

2.  The  two  integer  (1  byte  per  pixel)  images  are  then  loaded  into  Matlab.  An  EMD 
‘framework’  is  set  up  for  sub-sampling  the  image  in  two  dimensions,  onto  a 
retinotopic  array  based  on  the  ommatidial  separation  of  the  fly  eye.  At  present  the 
array  is  rectangular  to  aid  further  development  of  the  model  without  the  complexity 
of  a  hexagonal  lattice  as  present  in  the  fly,  but  elaboration  into  a  hexagonal  mosaic 
at  a  later  stage  should  be  straightforward,  since  this  can  be  modeled  as  a  pari  of 
interleaved  rectangular  mosaics  (with  one  offset  vertically  by  half  an 
‘ommatadium’.. 

3.  Based  on  the  retinotopic  framework,  the  inputs  to  a  single  row  of  ommatidia  in  time 
are  generated  from  a  single  row  of  the  image,  converted  to  floating-point  and  then 
animated  in  space-time  according  to  a  pre-defined  vector  of  velocities,  as  in  the  1-d 
model. 

4.  The  space-time  matrix  is  then  temporally  filtered  using  a  2-dimensional  convolution 
with  a  separable  kernel.  The  kernel  is  based  on  a  difference  of  log-normal 
functions,  to  simulate  the  impulse  response  of  a  fly  lamina  monopolar  cell  (LMC). 

5.  A  copy  of  the  temporally  fdtered  matrix  is  then  convolved  a  second  time  with  a 
kernel  to  implement  the  delay  fdter. 

6.  The  two  matrices  are  then  multiplied  after  column- wise  offset  by  a  single  subscript 
value  and  the  same  operation  is  carried  out  in  mirror  image. 
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7.  The  two  products  are  then  subtracted  to  produce  the  opponent  outputs  for  all  240 
EMDs.  This  matrix  is  then  added  to  a  3  dimensional  ‘stack’  matrix. 

8.  Additional  image  rows  are  processed  in  sequence  until  the  3  dimensional  matrix  is 
full. 

9.  Slices  across  the  resultant  output  matrix  can  be  considered  slices  at  single  instants 
in  time  for  the  output  of  all  EMDs  in  the  36  x  240  EMD  array. 

In  a  simulation  whose  results  are  detailed  below,  the  panorama  was  rotated  in  the  yaw  axis 
at  a  temporal  sample  rate  of  500Hz  for  8  seconds.  During  this  period,  the  pattern  was  is 
incremented  logarithmically  in  200  ms  bursts,  interspersed  with  bursts  of  an  adapting  speed. 
The  entire  simulation  took  approximately  4  minutes  to  generate  using  a  900Mhz  CPU  running 
Matlab. 

Figure  2  below  shows  the  mean  output  of  this  simulation,  averaged  for  all  EMDs  in  the 
240x36  array,  and  demonstrates  how  spatial  summation  across  the  panorama  reveals  the 
underlying  response  with  superb  rejection  of  pattern  noise.  During  the  early  part  of  the 
response  period,  responses  during  the  adapting  ‘bursts’  are  stronger  than  during  the  (low  speed) 
test  patterns,  but  as  speed  of  the  test  bursts  increases,  the  response  rises  to  a  maximum  (around 
2500  ms,  corresponding  to  a  burst  speed  of  ca.  300  degrees/s).  This  trace  already  mimics  many 
of  the  features  of  the  neurobiological  recordings  in  monotonic  experiments  (e.g.  figure  1), 
including  prominent  response  ‘transients’  at  the  change  of  speed  between  adapting  and  test 
bursts  of  motion. 


0  500  1000  1500  2000  2500  3000  3500  4000 

Time  (ms) 

Figure  32:  Spatially  averaged  EMD  responses  to  interleaved  ‘monotonic’  protocol  for 
panoramic  image  motion,  as  used  in  the  neurobiological  experiments. 
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We  initially  setup  this  model  using  no  adaptive  feedback,  but  then  refined  the 
neurobiological  model  as  reported  during  the  last  period  to  include  the  adaptive  feedback 
behavior.  Two  time-slices  from  the  output  matrix  of  a  simulation  of  the  adaptive  model  are 
shown  in  figure  33  below,  taken  from  a  matrix  movie  showing  the  evolving  output  of  the  2 
dimensional  EMD  array  over  time.  The  movie  itself  is  embedded  into  figure  34  (requires  the 
QuickTime  engine  to  run  -  a  separate  copy  of  the  movie  is  included  in  the  accompanying  CD). 
The  simulation  was  for  36  x  240  EMDs,  subtending  the  full  360  degree  panoramic  ‘scene’  in 
the  horizontal  dimension,  and  corresponding  approximately  to  the  field  of  view  of  the  two 
equatorial  HS  neurons  (HSE)  of  the  fly  motion  pathway. 


Figure  33:  Adaptive  EMD  responses  to  moving  natural  imagery. 

The  first  frame  shows  the  EMD  array  response  100ms  after  a  transition  in  speed  from 
motion  at  an  adapting  speed  (in  this  case  a  low  adapting  speed  of  5  degrees  per  second  was 
selected)  to  a  higher  test  speed  (150  degrees/s).  The  low  speed  of  the  adapting  pattern  produces 
little  response  of  EMDs  and  thus  little  motion  adaptation,  so  that  local  EMD  gain  is  in  the 
default  ‘high’  state,  as  observed  in  the  fly  under  similar  coniditions.  By  the  time  of  this  1st 
‘snapshot’,  local  EMDs  are  responding  strongly  to  the  onset  of  the  faster  pattern.  By  the  time  of 
the  second  frame  (100  ms  later  still)  local  motion  adaptation  has  selectively  reduced  the  gain  of 
local  EMDs  that  previously  viewed  high  contrast  features.  The  overall  gain  reduction  is 
apparent  from  the  general  decrease  in  EMD  activity.  One  interesting  observation  here  is  that 
high  contrast  features,  which  produce  the  largest  response  (and  greatest  local  adaptation)  are 
not  randomly  distributed  within  the  scene,  but  tend  to  be  clustered. 


Figure  34:  Animation  showing  adaptive  EMD  model  responses  to  moving  natural  imagery 
(as  in  fig  33).  Double  click  to  play  (requires  Apple  Quicktime  engine). 

Thus  for  optic  flow  of  the  kind  simulated  (in  this  case  ‘rigid’  yaw)  any  EMD  which  is 
initially  stimulated  strongly  (and  thus  adapts)  may  have  a  significant  effect  on  the  subsequent 
response,  since  additional  high-contrast  features  are  likely  to  pass  across  its  field  of  view  while 
it  remains  adapted.  One  such  case  is  illustrated  in  figure  33,  where  a  high-contrast  local  feature 
(circled)  is  ‘followed’  by  two  others.  Initially,  this  leads  to  3  distinct  ‘patches’  of  local 
excitation  (hot  colors),  but  following  adaptation,  the  rightmost  pair  of  these  features  (arrow) 
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produce  only  weak  excitation,  since  they  pass  across  the  adapted  EMD  initially  excited  by  the 
left-most  patch.  This  illustrates  the  fact  that  the  effect  of  adaptation  on  the  global  response  of  a 
collator  neuron,  such  as  an  HS  cell,  depends  on  a  complex  (and  non-linear)  interaction  between 
local  adaptive  properties  and  the  particular  statistical  distribution  of  contrasting  features  in  the 
image. 

It  is  difficult  to  make  empirical  predictions  about  how  such  a  complex  set  of  interactions 
will  behave  in  the  presence  of  more  complex  optical  flow  sequences,  such  as  might  be 
experienced  during  complex  behavior.  For  this  reason,  we  also  commenced,  during  this  period, 
construction  of  an  even  more  elaborated  and  generalized  class  of  EMD  models,  in  which  we 
would  be  able  to  supply  arbitrary  sequences  of  discrete-time  sampled  data  to  test  responses  to 
more  complex  optic  flow  (e.g.  combinations  of  translation  and  rotational  motion).  During  this 
period,  we  constructed  the  ‘front  end’  to  permitting  modeling  of  the  spatial  imaging  properties 
of  the  EMD  array  given  an  arbitrary  ‘movie’  as  the  data  source.  We  tested  this  with  optic  flow 
sequences  generated  by  flight  simulator  software  running  on  a  modem  accelerated  PC  graphics 
card.  One  problem  with  such  simulations  is  that  the  limited  frame  rate  at  which  such  data  can 
be  rendered  limits  the  temporal  sample  rate  for  simulations  based  on  their  output  in  real  time. 
‘Flying’  in  slow  motion  and  then  scaling  the  simulation  to  higher  speeds  would  enable  us  to 
simulate  effective  frame  rates  up  to  1000Hz,  sufficient  to  approximate  a  continuous-time 
system  with  time  constants  of  the  order  of  those  seen  in  the  biological  system.  However,  this 
approach  would  be  limited  to  simulation  of  relatively  fast  moving  scenes.  This  was  regarded  as 
sufficient  for  testing  model  EMD  array  responses  in  a  variety  of  interesting  scenarios,  but  we 
planned  to  eventually  move  to  more  explicit  rendering  of  3-D  motion  stimuli  in  order  to 
simulate  the  full  range  of  image  speeds  to  which  neurons  are  able  to  respond 

2.2.2  Year  2 

2.2.2. 1  Elaboration  of  EMD  output  stages 

With  the  neurobiological  experiments  suggesting  a  degree  of  'velocity  constancy',  attention 
in  modeling  turned  to  quantifying  the  behavior  of  the  adaptive  model  in  the  context  of  different 
images  from  the  set  in  figure  3.  Discussion  with  our  collaborators  over  the  concurrent  SBIR 
effort  considered  two  alternative  models,  one  based  on  feedforward  contrast  gain  and  one  using 
feedback  (as  modeled  above),  and  the  features  known  to  be  present  in  motion  adapted  neurons 
from  our  experiments.  We  concluded  that  a  model  based  on  a  feed-forward  contrast 
normalization  algorithm  may  not  be  able  to  explain  the  observation  that  gain  reduction  during 
brief 'bursts'  of  motion  at  a  variety  of  speeds  appears  greatest  at  those  speeds  where  HS  neurons 
generate  maximal  responses.  On  the  other  hand,  it  was  not  clear  how  an  alternative  mechanism, 
such  as  the  feedback  mechanism  described  above  which  is  based  on  direct  control  of  gain  by  a 
signal  fed  back  from  EMD  outputs,  could  produce  velocity  constancy  over  any  but  a  limited 
range  of  speeds.  The  issue  is  confounded  by  the  observation  that  the  biological  system  shows 
two  distinct  components  to  motion  adaptation,  one  of  which  being  a  gain  reduction  and  the 
other  (the  so-called  'waterfall  effect')  the  addition  of  an  apparent  DC  offset.  The  latter  is  only 
recruited  by  motion  in  the  optimum  direction,  whereas  gain  reduction  is  recruited  by  motion  in 
any  direction.  At  this  point,  our  biological  experiments  suggested  velocity  constancy  but  had 
all  been  carried  out  with  preferred-direction  stimuli.  We  concluded  that  further  neurobiological 
experiments  to  repeat  such  stimulus  trains  in  the  anti-preferred  direction  would  be  an  excellent 
way  to  tease  these  components  apart. 

Prior  to  completion  of  such  experiments,  the  'correct'  model  approach  was  undecided,  with 
both  feed-forward  and  feedback  models  having  both  desirable  and  undesirable  features.  We 
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agreed  that  the  feedforward  model  would  be  explored  in  detail  in  the  concurrent  SBIR  project, 
while  we  would  explore  further  elaborations  of  the  feedback  model  in  this  project.. 

A  set  of  simulations  was  therefore  performed  with  a  large  adaptive  EMD  array  operating  on 
natural  image  data,  which  suggest  that  a  gain  control  scheme  based  on  feedback  of  EMD 
outputs  can  result  in  at  least  some  degree  of  velocity  constancy.  In  addition  to  feedback,  this 
model  featured  biologically  plausible  prefiltering,  and  limiting  or  squashing  nonlinearities 
applied  at  several  stages  in  the  processing  chain,  including  a  tank  compression  of  the  individual 
EMD  outputs,  and  a  mild  compressive  nonlinearity  to  the  summed  EMD  outputs,  which 
represents  nonlinear  ‘HS  cell’  processing.  This  was  similar  to  the  compression  used  by  us  in 
earlier  modeling  efforts  (Dror  et  al  2000).  In  the  feedback  adaptation,  each  EMD  output  was 
rectified,  lowpass  filtered,  and  passed  through  a  decreasing  nonlinear  function  that  starts  at 
unity  for  small  arguments,  and  decreases  toward  zero  for  larger  ones.  Specific  functions  used 
included  0.5*(l-tanh[k(x-m)])  and  SQRT(l-bx),  where  x  is  the  argument  and  k,  m,  and  b  are 
constants.  (In  practice,  values  for  b  were  chosen  so  that  negative  for  values  of  the  argument  of 
the  square  root  did  not  occur.)  The  output  of  this  function  then  multiplied  the  inputs  to  the 
EMD  to  effect  the  adaptive  gain  reduction. 

Simulations  based  on  this  model  followed  the  ‘test-adapt-test’  protocol  used 
experimentally.  Some  example  outputs  are  shown  below  in  figure  35.  The  output  variable  is  the 
(compressed)  sum  of  all  EMD  outputs  in  the  array,  and  the  tuning  curves  are  generated  based 
upon  the  output  values  during  ‘test  phases’  at  various  image  velocities. 
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Figure  35:  Results  of  a  simulation  of  an  EMD  array  with  adaptive  feedback,  subject  to  the  test- 
adapt-test  protocol  used  to  determine  HS  cell  velocity  tuning.  The  output  is  a  (compressed)  sum  of 
the  individual  EMDs  in  the  array.  Nonlinear  feedback  uses  the  squareroot-based  function 
described  in  the  text.  Top  is  the  time  domain  response,  and  bottom  the  velocity  tuning.  Various 
contrasts  and  adapting  speeds  are  indicated  in  the  legends. 

The  model  shows  firstly  that  it  is  possible  to  obtain  a  reasonable  fit  to  the  observed  velocity 
tuning  of  HS  neurons  simply  by  choosing  parameters  for  key  filter  stages  measured  directly  in 
the  fly  visual  system.  Secondly,  it  shows  that  it  is  possible  to  explain  some  deviation  from  the 
predictions  of  the  classical  Reichardt  model  for  variable  contrast  using  a  feedback  gain  control. 
However,  when  tested  at  very  low  contrasts,  the  model  displays  the  expected  contrast 
dependence  (quadratic)  resulting  from  the  multiplication  mechanism. 

These  simulations  suggested  that  a  gain  control  scheme  based  on  feedback  of  EMD  outputs 
can  result  in  at  least  some  degree  of  velocity  constancy.  We  found  the  model  behavior  to  be 
strongly  affected  by  the  image-prefiltering  applied.  We  began  exploring  possible  alternatives  to 
replacing  simple  logarithmic  compression  at  the  ‘photoreceptor’  level  (which  provides  a  poor 
model  for  the  behavior  of  real  photoreceptors  at  very  high  luminances,  or  of  the  Delbriick  and 
Mead  -type  adaptive  photocircuit  that  our  hardware  efforts  utilize)  with  more  biologically 
plausible  non-linear  compression. 

2. 2. 2. 2  Elaborations  of  delay  filter  element 

The  'baseline'  EMD  model  used  at  this  point  incorporated  a  relatively  simple  first-order 
low-pass  filter  as  the  delay  element.  However,  additional  work  around  this  time  (O'Carroll  in 
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conjunction  with  former  PhD  student  Rob  Harris)  suggested  that  a  simple  delay  model  might 
be  inadequate  to  explain  the  shape  of  observed  HS  neuron  temporal  frequency  tuning  curves.  In 
a  related  project  (Navy  funded,  N00014-02-M-0247)  we  explored  the  use  of  higher-order  delay 
filters  based  on  a  log-normal  model,  and  found  that  these  explain  several  features  of  the 
temporal  tuning  observed  in  nocturnal  moth  motion  detectors.  During  a  visit  by  former  Ph.D. 
student  Rob  Harris  to  the  O'Carroll  laboratory,  we  considered  similar  delay  filters  for  fly 
motion  detectors.  Our  new  models  show  excellent  fits  of  such  filters  to  fly  temporal  tuning,  so 
this  type  of  delay  mechanism  may  be  widespread  within  insects.  Temporal  tuning  so  derived  is 
characterized  by  rapid  roll-off  at  high  frequencies,  suggestive  of  a  relatively  high  filter  order, 
and  very  broad  temporal  tuning.  We  have  found  that  in  modeling  Calliphorid  fly  HS  responses, 
a  complex  delay  filter  consisting  of  two  such  delay  elements  in  parallel  provides  a  better  fit 
than  a  single  delay  model. 

In  subsequent  modeling,  we  tried  to  produce  implementable  forms  of  this  lognormal  delay 
mechanism.  Using  Prony's  method,  we  fitted  arbitrary  infinite  impulse  response  filters  to  the 
'ideal'  lognormal  impulse  response,  and  found  that  we  were  able  to  mimic  its  characteristic 
time-course  by  filters  of  6th  order.  We  planned  further  analysis  of  these  filters  with  a  view  to 
deriving  a  transfer  function  with  discrete  poles  that  would  be  useful  for  biomimetic 
implementation. 

2.2.23  Modeling  receptive  field  shape  to  reduce  pattern  noise  effects 

We  further  quantified  the  effects  of  receptive  field  shape  on  pattern  noise,  measured  as 
'relative  error'  (see  Dror,  O'Carroll  &  Laughlin  2001,  J.  Opt.Soc.Am.A  18:241-252).  We 
defined  this  error  as  the  residual  variance  over  time  in  the  response  due  to  the  local  structure  of 
the  image,  divided  by  the  mean  (DC  level)  of  that  response.  The  assumption  here  is  that  the  DC 
level  of  the  summed  outputs  of  an  EMD  array  represents  the  desired  velocity  signal,  while 
time-variation  due  to  the  pattern  structure  is  effectively  noise,  so  that  relative  error  is  a  measure 
of  signal-to-noise  ratio  in  the  context  of  the  task  of  velocity  coding. 

Our  hypothesis  was  that  the  semi-circular  (fully  circular  if  summation  at  the  steering 
muscles  is  taken  into  account),  strip-like  nature  of  the  HS  cell  receptive  field  plays  a  functional 
role  in  encoding  velocity.  Because  the  circular  array  of  EMDs  is  parallel  to  the  preferred 
direction  of  motion,  a  purely  circular  optical  flow  pattern  (i.e.  yaw  for  an  HS  neuron)  would 
produce  the  same  temporal  sequence  of  stimulation  of  individual  photoreceptors.  Hence  if  a 
high  contrast  feature  were  present  within  the  receptive  field  at  one  point  in  time,  it  would  also 
be  very  likely  to  be  present  at  future  times.  By  summing  across  such  an  array,  pattern  noise 
would  be  largely  reduced. 

To  test  this  hypothesis,  we  ran  simulations  of  the  2-d  model,  with  an  array  size  of  240  by  37 
EMDs,  each  with  an  inter-receptor  angle  of  1.5  degrees,  and  subtending  a  total  of  360  degrees 
of  visual  angle.  We  ‘stimulated’  the  array  with  a  panoramic  yaw  stimulus  with  a  fixed  velocity, 
for  2  seconds.  We  ran  the  simulation  for  4  different  panoramic  images  and  12  different  speeds 
for  each.  We  then  sampled  the  matrix  of  EMD  outputs  in  one  of  4  ways.  The  first  (labeled 
‘total’  in  the  figures  below)  is  the  sum  (across  the  time  dimension)  of  all  EMDs.  The  second 
used  a  circular  linear  array  with  just  240  EMDs,  representing  one  of  the  37  possible  rows  of 
EMDs,  selected  at  random.  The  third  was  a  randomly  selected  rectangular  grid  of  15x16  EMDs 
(i.e.  240  total).  The  4th  consisted  of  240  EMDs  selected  randomly  from  the  240x37  array.  In 
each  of  these  subsampled  EMD  sets,  we  repeated  our  estimate  20  times,  using  a  different 


n  AVm  O’P  ARBDT  T  T  TnTVPR  STTV  OF  AnPI  ATTYP 


AOARD-03-4026  /  RMO  No  2001/161 1 


64 


random  number  seed  in  each  case.  Relative  error  (as  defined  above)  was  then  estimated  for  the 
linear  sum  of  the  EMDs  in  each  sample  and  averaged  across  the  20  trials. 


Mean  relalltfe  errors,  all  images  Mean  EMC1  OuLpuL  L  Standard  deviation 


Angular  yHfldtj'  |dag'fi|  vefcHitir  (dflgft) 


Figure  36  Relative  errors  (left)  and  average  outputs  (right)  of  several  array  topologies  as 
defined  in  the  text.  Modeling  By  O’Carroll  and  Sreeja  Rajesh  (PhD  student,  O’Carroll 
lab) 

As  can  be  seen,  the  circular  linear  array  achieved  a  very  similar  variance  in  velocity 
estimation  to  that  of  the  ‘full’  EMD  set,  and  (depending  on  speed)  10  or  more  times  lower  than 
the  other  two  strategies  for  sampling  EMDs.  We  concluded  that  a  circular,  linear  detector  array 
may  be  the  ideal  and  most  efficient  way  to  sample  outputs  from  EMDs  of  the  correlator  type,  at 
least  with  respect  to  reduction  of  pattern  noise  in  rotational  motion.  This  suggested  that  a  small 
detector  array  with  as  few  as  240  pixels  could  provide  robust  estimates  of  speed  under  natural 
image  conditions. 

2. 2. 2. 4  Elaboration  of  input  compressive  nonlinearity 

Further  modification  was  made  to  the  2-d  model  to  add  elaborations  to  the  scaling  of 
luminance  in  the  model  inputs.  Our  previous  modeling  efforts  made  the  assumption  that 
biological  photoreceptors  encode  luminance  logarithmically.  Our  elaborated  EMD  model 
showed  (desirable)  reduced  sensitivity  to  image  contrast  when  we  placed  saturation  at  any  of 
several  locations  in  the  EMD.  Since  strict  logarithmic  encoding  is  non-saturating  (and 
biologically  unrealistic),  we  thus  began  working  with  implementation  of  the  Naka-Rushton  (or 
Lipetz)  equation: 

R  =  I"  /(X"  +In), 

where  /  is  the  raw  luminance  value,  X  is  the  intensity  giving  50%  response  (midpoint  for  the 
dynamic  range)  and  n  is  the  slope  of  the  ‘linear’  region.  Values  for  n  were  selected  to  give  a 
logarithmic  intensity  code  over  two  base- 10  log  units  of  intensity.  This  curve  provides  a 
sigmoidal  (on  log  intensity  scale)  relationship  between  input  and  output  that  closely  fits  the 
observed  V/Log(I)  curves  of  biological  photoreceptors.  Incidentally,  the  same  equation  gives 
an  excellent  fit  to  the  V/Log(I)  function  for  adaptive  photocircuits  of  the  Delbriick  and  Mead 
type,  as  implemented  in  our  silicon  models. 
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2. 2. 2. 5  Modeling  the  waterfall  effect 

Thus  far,  no  attempt  had  been  made  to  ‘fit’  the  amount  of  adaptive  behavior  in  the  model  to 
any  of  the  observed  biological  data.  Modeling  efforts  thus  turned  to  using  the  HS  data  obtained 
in  our  earlier  work  (Harris,  O'Carroll,  Laughlin,  2000,  Contrast  gain  reduction  in  fly  motion 
adaptation,  Neuron  28:  595-606).  The  experiment  in  that  paper  consisted  of  the  test-adapt-test 
protocol.  This  was  implemented  in  a  model  as  a  variable  contrast  test  pattern  presented  for  1 
second  before  and  after  a  4  second  adapting  stimulus.  The  adapting  pattern  had  maximal 
contrast,  high  temporal  frequency  (20Hz)  and  low  spatial  frequency  (0.02  cycles/degree). 
Although  a  'non-optimaf  stimulus  for  the  HS  cell,  this  stimulus  produces  powerful  motion 
adaptation  and  is  fast  enough  that  it  avoids  confounding  'afterimage'  effects.  We  used  this  as  a 
standard  adapting  stimulus  during  all  of  our  earlier  experiments. 

The  responses  were  evaluated  during  a  100ms  'window'  25ms  after  the  commencement  of 
the  test  stimulus,  and  then  plotted  against  the  contrast  of  the  test  pattern. 

In  four  ‘virtual  experiments’  designed  to  mimic  the  experiments  in  the  previous 
neurobiological  study,  we  varied  the  test  and  adapting  patterns  in  all  four  possible 
combinations  of  preferred  and  antipreferred  directions.  The  direction  selectivity  of  the  motion 
aftereffect  (waterfall  effect)  was  very  evident  in  the  neurobiological  data  as  a  downward 
(negative)  shift  in  the  control  response  (lowest  contrast)  for  post-adaptation  responses.  Note 
that  this  was  only  observed  when  the  adapting  stimulus  was  moving  in  the  preferred  direction 
(as  seen  in  the  top  left  and  bottom  right  panels  in  Figure  37  below).  In  all  four  cases  there  was  a 
pronounced  contrast  gain  reduction,  evident  as  a  rightward  shift  in  the  entire  contrast  response 
function  (increased  contrast  threshold).  Other  experiments  (not  shown)  demonstrated  that  this 
effect  is  motion  dependent. 

We  modeled  this  system  as  circular  array  of  240  EMDs,  viewing  a  sinusoidal  pattern  that 
subtends  360  degrees  and  was  assumed  to  be  panoramic,  i.e.,  the  pattern  contained  an  integer 
number  of  stimulus  cycles,  matched  to  that  used  in  the  experiments.  Each  EMD  within  the 
array  was  elaborated  (as  described  in  previous  reports)  by  inclusion  of  biologically  inspired 
elements.  The  image  was  low-pass  filtered  by  a  Gaussian  kernel  of  2-degree  half-width,  then 
animated  to  mimic  the  exact  sequence  of  stimuli  in  the  HS  experiments.  The  model  included  a 
saturating  logarithmic  'photoreceptor'  stage,  modeled  using  the  Lipetz  (or  Naka-Rushden) 
equation.  The  slope  n  in  the  log-linear  region  was  set  to  give  a  dynamic  range  (logarithmic 
region)  spanning  2  log  units  of  intensity,  as  observed  in  recordings  from  typical  diurnal 
apposition  eyes  such  as  that  in  the  fly. 

In  the  case  of  the  sinusoidal  images  used  in  these  simulations,  the  images  were  scaled  to 
range  from  0-255,  with  a  mean  of  128,  in  order  to  match  the  range  of  luminance  values  in 
typical  8-bit  images.  This  allowed  the  same  model  to  be  applied  to  any  other  arbitrary  image, 
including  the  panoramic  natural  scenes  used  in  other  concurrent  experiments.  After 
transformation  through  the  Naka-Rushden  curve,  the  image  is  rescaled  to  a  range  from  0-1. 

The  animated  image  was  temporally  band-pass  filtered  by  a  linear  kernel  based  on  the 
double  log-normal  equation  fitted  to  Eristalis  lamina  monopolar  cell  (LMC)  responses.  EMDs 
were  then  constructed  using  multiplication  of  the  pre-filtered  data  with  a  delayed  version  (first- 
order  low-pass,  time  constant=35  ms).  The  multiplication  on  each  EMD  subunit  included  a 
saturation  mechanism  (based  on  a  simple  tank  transformation  of  the  output).  Each  EMD 
incorporated  two  adaptive  mechanisms.  The  first  was  a  feedback  gain  control,  as  described 
earlier.  The  second  was  a  direction-selective  offset,  applied  as  a  linear  function  of  depolarizing 
EMD  outputs.  In  this  version  of  the  model,  this  was  applied  locally  to  each  EMD,  based  on  the 
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half-wave  rectified  EMD  output,  after  low-pass  filtering  (1st  order,  0.2Hz  corner  frequency). 
An  alternative  form,  where  this  offset  is  applied  to  the  low-pass  filtered  sum  of  all  EMDs  in  the 
array  (a  'global'  mechanism)  would  be  equally  plausible  for  the  biological  system  and  could  not 
be  excluded  based  on  any  experimental  results  obtained  to  this  point.  These  two  alternatives 
would  have  similar  quantitative  effects. 

After  subtraction  of  the  two  mirror  symmetrical  subunits  in  each  EMD,  local  EMDs  were 
then  summed  using  a  non-linear  spatial  summation  mechanism,  here  modeled  as  a  simple 
saturation  in  the  summation  mechanism. 

Simulations  were  run  for  the  same  set  of  test  contrasts  as  used  in  HS  recordings.  The  gain 
of  the  feedforward  stages  was  set  to  match  the  contrast  threshold  (contrast  required  to  evoke  a 
50%  maximal  response)  in  the  unadapted  response  to  stimuli  moving  in  the  preferred  direction 
(i.e.  the  data  illustrated  by  square  symbols  in  the  top  left  panel  of  the  figure).  The  gain  of  the 
feedback  loop  that  acts  to  reduce  contrast  sensitivity  following  motion  adaptation  was  then 
adjusted  to  match  the  right-shift  in  contrast/response  functions  following  preferred  direction 
adaptation.  An  additional  adjustment  was  made  to  match  the  observed  DC  offset  (the  motion 
aftereffect).  Note  that  apart  from  adjustment  of  these  two  parameters,  no  explicit  attempt  was 
made  to  fit  the  dynamic  range  of  the  contrast  sensitivity  curves:  The  observed  excellent  fit  in 
both  cases  was  due  entirely  to  the  quadratic  nature  of  the  essential  non-linearity  of  the  local 
motion  detectors  (i.e.  the  multiplication  stage),  the  dynamic  range  of  the  underlying 
photoreceptors  (modeled  closely  on  that  observed  in  real  recordings  from  insect 
photoreceptors),  and  the  various  saturation  stages  built  into  the  model,  each  of  which  was  based 
on  published  or  measured  values  from  the  physiological  data. 

The  same  model  was  then  applied  to  the  other  three  experimental  conditions.  This  model 
provided  a  surprisingly  good  fit  to  several  interesting  phenomena  in  the  physiological  data. 
Firstly,  the  model  nicely  captured  the  response  range  compression  and  slope  reduction  noted  in 
null-direction  test  following  preferred  direction  adaptation  (bottom  right  in  Figure  1),  where  the 
DC  offset  (motion  aftereffect)  in  effect  adds  to  the  response  to  test  stimuli.  It  failed,  however, 
to  produce  pronounced  response  range  compression  in  other  situations,  suggesting  that  contrast 
gain  reduction  may  be  weaker  in  the  biological  system  than  in  this  'best  fit'  model.  It  is  likely 
that  this  deficiency  could  be  addressed  by  a  more  'biomimetic'  implementation  of  the  non-linear 
saturation  mechanism  in  the  spatial  summation  stage,  to  better  match  the  biophysical  properties 
of  the  HS  dendritic  tree.  This  may  actually  be  simpler  in  a  hardware  implementation  of  this 
model  than  in  the  present  (Matlab-based)  simulation. 
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Pattern  contrast  Pattern  contrast 

Figure  37:  Experimental  and  modeled  contrast  sensitivity  curves  for  combinations  of 
preferred-  and  antipreferred  (null)-direction  adapting  and  test  stimuli.  Both  actual  and 
modeled  results  are  depicted. 


2.2.3  Year  3 

2. 2. 3. 1  Analysis  of  Results  to  Date 

At  this  point  in  the  effort,  we  collated  and  examined  the  modeled  tangential  cell  responses 
in  work  to  this  point.  While  many  experimentally  observed  characteristics  had  been 
reporduced,  none  of  the  biologically  plausible  models  had  achieved  the  degree  of  velocity 
constancy  seen  in  the  biological  neuron  with  natural  imagery.  While  a  high  degree  of  velocity 
constancy  could  be  obtained  with  very  high  gain  and  saturating  nonlinearity  at  the  inputs  to  the 
multiplication  stages  of  the  EMD,  such  models  would  be  expected  to  perform  poorly  in  the  real 
world,  since  they  would  be  susceptible  to  noise  about  the  threshold  value,  which  must,  by 
definition,  lie  close  to  the  mean  luminance  in  a  given  image.  Velocity  constancy  was  only 
achieved  by  saturating  at  contrasts  below  those  observed  in  the  natural  scenes  that  we  had 
sampled  to  date;  with  these  parameters,  saturation  in  response  to  spatially  narrow-band  scenes 
(i.e.  sinusoidal  gratings)  was  also  observed  at  contrasts  much  lower  than  those  at  which 
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complete  saturation  is  observed  in  the  biological  system.  Furthermore,  the  saturating  model 
failed  to  display  one  of  the  most  striking  observations  of  the  biological  data:  reducing  the 
contrast  of  the  natural  image  by  re-scaling  its  luminance  distribution  produces  a  reduction  in 
response  to  any  one  image  (whereas  the  highly  saturating  model  would  not)  and  yet  the 
neurons  still  display  robust  velocity  constancy  with  respect  to  different  images. 

Simulations  were  run  in  which  both  components  of  adaptation  -  post-stimulus  changes  in 
membrane  potential  (the  ‘waterfall  effect’)  and  contrast  gain  reduction  -  were  incorporated  in  a 
feedback  adaptive  model.  This  elaboration  was  observed  to  result  in  velocity  constancy  for 
natural  imagery  in  preferred-direction  motion.  However,  due  to  symmetry  of  the  gain-reduction 
part  of  the  model  and  the  fact  that  the  waterfall  effect  is  not  observed  in  antipreferred  direction 
motion,  we  did  not  expect  velocity  constancy  to  hold  for  antipreferred  direction  stimuli  (as  it 
does  in  the  biological  neuron). 

Further  elaborations  of  the  model  were  deferred  to  our  concurrent  SBIR  effort,  while  the 
remainder  of  this  project  focused  on  in-depth  analysis  of  possible  physiological  drivers  for 
adaptation.  Again,  the  primary  conclusion  drawn  from  the  results  of  this  study  was  that,  while 
all  of  the  test  features  reduced  interscene  scatter  of  ‘tangential  cell’  outputs  relative  to  the  most 
basic  version  of  the  model,  in  no  case  did  the  model  approach  velocity  constancy  as  closely  as 
does  the  biological  system.  Combining  our  efforts  from  both  AOARD  and  SBIR  projects,  we 
submitted  a  summary  of  our  model  outputs  as  a  paper  which  was  subsequently  accepted  for 
publication  by  the  journal  Biological  Cybernetics.  The  key  findings  are  illustrated  below  in 
figure  38. 
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HS  Cell  Average  Outputs 
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Figur  38:  Scatter  of  responses  between  different  animated  scenes  of  the  model  in  various 
configurations,  at  the  near-optimal  speed  of  50°/s.  In  each  column  of  the  graph  are 
displayed  responses  to  a  set  of  five  different  images  for  a  particular  model  configuration, 
normalized  by  the  mean  over  all  five.  Configurations  include  each  of  the  test  features 
individually:  1.  basic  model;  2.  with  spatial  highpass  filtering;  3.  with  motion  adaptation 
(gain  control)  with  time  constant  xA  =  200ms;  4.  with  saturation  of  early  vision  signals;  5. 
with  saturation  of  EMD  correlator  outputs;  6.  with  Borst  ‘gain  control’  in  the  HS  cell. 
Column  7  shows  mean  responses  of  biological  HS  cells  in  the  hoverfly  Eristalis  tenax  to 
yaw  stimuli  with  the  three  of  the  images,  at  a  speed  of  1007s. 


Also  shown  in  Figur  are  three  data  obtained  from  biological  HSNE  cells  in  the  hoverfly 
Eristalis  tenax,  for  the  images  hamlin,  close,  and  gardens  in  simulated  yaw  rotation  at  1007s. 
Experimental  preparation  was  as  described  in  Harris  et  al.  (2000).  Results  are  averaged  over 
the  time  period  0.25s  to  0.4s  following  onset  of  motion  at  1007s  (preceded  by  motion  stimuli  at 
slightly  different  speeds),  over  four  stimulus  presentations  at  four  phases  varying  by  90°,  and 
over  five  neurons. 

Of  the  nonlinear  test  features,  the  simple  gain-control-based  motion  adaptation  model 
provided  the  greatest  reduction  in  interscene  scatter  relative  to  the  baseline  model.  Spatial 
highpass  filtering  significantly  improved  the  performance  of  the  model  in  combination  with 
one  or  more  of  the  nonlinear  features,  although  it  resulted  in  little  reduction  in  scatter  by  itself. 
This  we  suggested  is  due  to  reduction  of  the  amplitude  of  low  spatial  frequency  components  of 
natural  imagery,  which  tend  to  have  large  interscene  variance.  Finally,  we  found  that 
combining  multiple  nonlinear  features  did  not  generally  improve  performance. 

2.3  Milestones  and  Metrics 

2.3.1  Publications 

(*  asterisks  denote  work  partially  or  fully  funded  by  this  AOARD  project_ 
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2. 3. 1.1  Refereed  Papers 

*Shoemaker,  P.A.,  O’Carroll,  D.C.  and  Straw,  A.  (2005)  Velocity  constancy  and  models 
for  wide-field  motion  detection  in  insects.  Biological  Cybernetics,  (in  press: 
http://dx.doi.org/10.1007/s00422-005-0007-y). 

*Mah,  E.L.,  Brinkworth  R.  and  O'Carroll,  D.C.  (2005)  Bio-inspired  analog  circuitry  model 
of  insect  photoreceptor  cells.  Proc.  SPIE  5839  (in  press). 

*Rajesh,  S.,  Straw,  A.,  O'Carroll,  D.C.  and  Abbott,  D.  (2005)  Effect  of  spatial  sampling  on 
pattern  noise  in  insect-based  motion  detection.  Proc.  SPIE  5649,  p.  811-825 

*Rajesh,  S.,  Straw,  A.,  O'Carroll,  D.C.  and  Abbott,  D.  (2005)  Effects  of  compressive 
nonlinearity  on  insect-based  motion  detection.  Proc.  SPIE  Vol.  5649,  p.  798-810 

*Rajesh,  S.,  Straw,  A.,  Rainsford,  T.  and  O'Carroll,  D.C.,  (2005)  Modeling  pattern  noise  in 
responses  of  fly  motion  detectors  to  naturalistic  scenes.  Proc.  SPIE  Vol.  5651,  p.  160- 
173 

Mah,  E.L.  and  O'Carroll,  D.C.  (2005)  Implementation  of  an  adaptive  photodetector  circuit 
inspired  by  insect  visual  systems.  Proc.  SPIE  Vol.  5649,  p.  839-848 

*Rajesh,  S.,  O'Carroll,  D.C.  and  Abbott,  D.  (2004)  Effects  of  nonlinear  elaborations  on  the 
performance  of  a  Reichardt  correlator.  Proc.  SPIE  Vol.  5275,  p.  287-303 

*Rajesh,  S.,  O'Carroll,  D.C.  and  Abbott,  D.  (2003)  Velocity  estimation  and  comparison  of 
two  insect-vision-based  motion-detection  models.  Proc.  SPIE  Vol.  5062,  p.  401-412 

*Rajesh,  S.,  O'Carroll,  D.C.  and  Abbott,  D.  (2002)  Elaborated  Reichardt  correlators  for 
velocity  estimation  tasks.  Proc.  SPIE  Vol.  4937,  p.  241-253 

2.3. 1.2  Conference  Presentations  and  Posters 

2005 

*Moyer  de  Miguel,  I,  Nordstrom,  K,  O’Carroll,  D  C.  “Local  motion  adaptation  in  insect 
visual  systems.”  Society  for  Experimental  Biology  Annual  Main  Meeting,  Barcelona, 
Spain.  2005  (pending) 

Nordstrom,  K  and  O’Carroll,  D  C.  “Asymmetric  receptive  fields  of  target  neurons  in  the 
dragonfly  lobula.”  Society  for  Experimental  Biology  Annual  Main  Meeting 
Barcelona,  Spain  2005  (pending) 

*Shoemaker,  P.A.,  O’Carroll,  D.C.  (2005)  Insect-based  visual  motion  detection  with 
contrast  adaptation.  To  appear,  Proc.  SPIE  Vol.  5783,  Defense  and  Security 
Symposium,  Infrared  Technology  and  Applications  Conference  (Orlando,  FL,  March- 
April  2005). 

2004 

*0’CaiToll,  DC,  Shoemaker,  PA,  Rainsford,  T,  Straw,  AD,  Mah,  EL  and  Rajesh,  S.  Motion 
detection  and  velocity  coding  of  natural  scenes  by  the  insect  visual  system.  Invited 
talk,  International  Conference  on  Intelligent  Sensors,  Sensor  Networks  and 
Information  Processing  (ISSN IP)  (Melbourne,  December  2004). 
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*Shoemaker,  P.A.  and  O’Carroll,  D.C.  (2004)  Silicon  and  abstract  modeling  of  elementary 
motion  detection.  Workshop  on  Insect  Sensors  and  Robotics  (Brisbane,  August  2004). 

*0’Carroll,  DC,  Shoemaker,  PA,  Rainsford,  T,  Straw,  AD,  Mah,  EL  and  Rajesh,  S.  Motion 
detection  and  velocity  coding  of  natural  scenes  by  the  insect  visual  system.  Invited 
talk,  Workshop  on  Insect  Sensors  and  Robotics  (Brisbane,  August  2004). 

*Mah,  EL  &  O’Carroll,  DC,  Implementation  of  Biomimetic  Models  for  Adaptive 
Photoreceptors  Based  on  Insect  Neurobiology.  Poster,  Workshop  on  Insect  Sensors 
and  Robotics  (Brisbane,  August  2004). 

*Rainsford,  T,  &  O’Carroll,  DC  Pattern  Noise  in  Neural  Coding  of  Natural  Moving 
Images.  Talk,  Workshop  on  Insect  Sensors  and  Robotics  (Brisbane,  August  2004). 

*Dowd,  L.,  Ali,  I.,  Straw,  AD  &  O’Carroll,  DC  Interfacing  VisionEgg-based  Stimulus 
Generators  for  Testing  Responses  of  Biological  and  Artificial  Vision  Systems.  Poster, 
Workshop  on  Insect  Sensors  and  Robotics  (Brisbane,  August  2004). 

Nordstrom,  K.  &  O’Carroll,  DC  Detection  of  small  low-contrast  targets  across  cluttered 
and  moving  backgrounds.  Poster,  Workshop  on  Insect  Sensors  and  Robotics 
(Brisbane,  August  2004). 

*Rainsford,TJ,  Moyer,  IM,  Browning,  M,  &  O’Carroll,  DC.  Does  the  neuroanatomy  of 
visual  neurons  in  hoverfly  match  visual  ecology?  (poster)  XXII  International  Congress 
of  Entomology,  Brisbane,  August  2004 

Nordstrom,  K  and  O’ Carroll,  D  C.  “Characterization  of  small  motion  detection  neurons  of 
Eristalis  tenax.”  XXII  International  Congress  of  Entomology,  Brisbane,  Australia, 
abstract  S12F57  +  PS2S1205.  Brisbane,  August  2004 

Nordstrom,  K,  Swan,  L,  Mayrhofer,  S  and  O’Carroll,  D  C.  “Comparisons  of  the  image-step 
response  in  flies  and  humans.”  XXII  International  Congress  of  Entomology,  Brisbane, 
Australia,  abstract  PS2S1209.  Brisbane,  August  2004 

*0’Carroll  DC  &  Rainsford,  TJ  What  drives  adaptive  gain  control  in  insect  motion 
detection.  Invited  talk,  XXII  International  Congress  of  Entomology,  Brisbane, 
Australia,  August  2004 

Nordstrom,  K  and  O’Carroll,  D  C.  “Small  target  motion  detector  neurons  are  present  in 
female  hoverflies.”  The  7th  Congress  of  the  International  Society  for  Neuroethology, 
Nyborg,  Denmark,  July  2004,  abstract  P0163 


2003 

*0’CaiToll,  D.C.,  Straw,  A.D.,  &  Shoemaker,  P.A.  Adaptive  gain  control  in  insect  motion 
detection.  In  N.  Eisner  &  H.  Zimmermann,  Eds.,  The  Neurosciences  -  From  Basic 
Research  to  Therapy:  Proceedings  of  the  29th  Gottingen  Neurobiology  Conference 
(Germany),  Georg  Thieme,  Verlag,  Stuttgart,  pp.  546-547.  (2003) 

DuBois,  RA.,  O’Carroll,  D.C.,  &  Shoemaker,  P.A.  Spatio-temporal  tuning  for  small  targets 
from  a  simulated  array  of  elementary  motion  detectors.  In  N.  Eisner  &  H. 
Zimmermann,  Eds.,  The  Neurosciences  -  From  Basic  Research  to  Therapy: 
Proceedings  of  the  29th  Gottingen  Neurobiology  Conference  (Germany),  Georg 
Thieme,  Verlag,  Stuttgart,  pp.  547-548.  (2003) 
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*Straw,  AD  &  O’Carroll,  D.C.  Ghosting  and  aliasing  artifacts  in  apparent  motion  displays 
eliminated  with  motion  blur.  In  N.  Eisner  &  H.  Zimmermann,  Eds.,  The 
Neurosciences  -  From  Basic  Research  to  Therapy:  Proceedings  of  the  29th  Gottingen 
Neurobiology  Conference  (Germany),  Georg  Thieme,  Verlag,  Stuttgart,  pp.  601-602. 
(2003) 

*Straw  AD,  O’Carroll  DC  Robust  velocity  estimates  by  fly  motion  detectors  following 
motion  adaptation  .  Annual  meeting,  Association  for  Research  in  Vision  and 
Ophthalmology  ,  Fort  Lauderdale  USA.,  2002.  Published  in:  Investigative 
Ophthalmology  &  Visual  Science  44:  4131  Suppl.  2  May  2003 


2002 

*Straw  AD,  O'Carroll  DC  Adaptation  of  wide-field  movement  detecting  neurons  in 
hoverflies  produces  velocity  constancy  to  natural  scenes.  Annual  meeting,  Vision 
Science  Society,  Sarasota,  Florida,  2002.  Published  in  Integrative  &  Comparative 
Biology  42  (6):  1320-1320  DEC  2002 

*Straw,  A.D.,  O’Carroll,  D.C.,  &  Shoemaker,  P.A.  (2002)  Adaptive  Motion  Detectors 
Inspired  by  Insect  Vision.  Proceedings  of  the  First  World  Congress  on  Biomimetics  and 
Artificial  Muscles  (  Albuquerque,  NM,  December  2002). 


2.3.2  Students  and  Post-Doctoral  Fellows  Supported  and  Advised 

Andrew  Straw:  Ph.D.  student.  Supported  by  a  Howard  Hughes  Medical  Institute  Pre- 
doctoral  Fellowship,  but  advisorship  supported  by  SBIR  and  IRI  (AOARD)  funds.  Worked 
extensively  on  the  study  of  velocity  constancy  in  coding  of  velocity  in  natural  scenes. 
Developed  the  VisionEgg  software  that  is  now  key  to  current  and  future  experimental  work  in 
this  area.  Received  the  Ph.D.  degree  in  August  2004;  currently  a  post-doctoral  fellow  in  the 
Dickinson  laboratory  at  California  Institute  of  Technology. 

Roger  DuBois:  Postdoctoral  Research  Associate.  Dec  2001-2003.  Supported  by  AFOSR 
funds  to  work  on  target  detection,  but  also  contributed  to  the  SBIR  and  AOARD  efforts  by 
helping  develop  optimized  filter  routines  in  C  for  compilation  into  Matlab  models. 

Tamath  Rainsford:  Postdoctoral  fellow.  Supported  by  IRI  (AOARD)  and  SBIR  funds  Feb 
2003  -  August  2004.  Was  initially  to  work  on  an  AFOSR  project,  but  subsequently  worked  on 
HS  cell  recordings  for  the  AOARD  effort.  Her  main  role  in  the  project  was  to  'flesh  out'  the 
numbers  of  recordings  for  the  velocity  constancy  data  for  Straw's  thesis  work  (hence  her  co¬ 
authorship  of  the  primary  manuscript  on  this  topic  (initially  rejected  by  Current  Biology  but  is 
currently  under  revision)).  She  also  performed  a  suite  of  experiments  looking  at  the  effect  of 
adapting  speed  and  adapting  contrast  on  the  'right  shift'  of  contrast  sensitivity  curves  (contrast 
gain  reduction),  using  natural  images.  These  experiments  were  incomplete  at  the  time  that  she 
accepted  a  longer-term  position  in  the  T-Ray  lab  (with  Derek  Abbott)  at  the  University  of 
Adelaide  in  July  2004. 
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Irene  Moyer:  Paid  part-time  using  SBIR  funds  after  Rainsford’s  departure,  in  order  to 
complete  recordings  left  unfinished.  She  has  subsequently  started  looking  at  the  influence  of 
the  'waterfall  effect'  component  of  adaptation  on  contrast  sensitivity,  using  prolonged  test 
stimuli  and  a  modification  of  the  probe-adapt-probe  stimulus  using  a  'strip-surround-strip' 
stimulus,  that  invokes  the  waterfall  effect,  but  not  local  contrast  gain  control.  These 
experiments  are  nearing  completion.  Despite  termination  of  the  SBIR  project,  she  will  continue 
work  under  other  support  (carried  forward)  until  these  questions  have  been  adequately 
addressed  to  make  a  definitive  paper. 

Russell  Brinkworth:  Post-Doctoral  Fellow.  Commenced  full-time  in  January  2005  under 
AFOSR  funding.  He  has  also  played  a  valuable  support  role  in  the  SBIR  project  since  that  time 
and  has  become  heavily  involved  in  construction  of  the  optical  head  for  the  Yaw  Sensor  as  well 
as  developing  grant  proposals  to  local  (ARC)  sources  to  continue  work  in  this  area. 

Sreeja  Rajesh:  Ph.D.  student.  Under  stipend  support  from  the  University,  but  IRI  funds 
have  supported  advisorship  (i.e.  O’Carroll’s  salary)  and  research  infrastructure  used  in  her 
work.  She  is  completing  a  PhD  project  modeling  pattern  noise  and  the  effects  of  non-linear 
components  of  EMDs  in  a  'yaw  simulation'  of  the  HS  receptive  field. 

Eng  Leng  Mah:  Ph.D.  student.  Supported  by  a  University  of  Adelaide  scholarship,  but 
SBIR/IRI  funds  again  have  supported  advisorship  and  research  infrastructure  used  in  his  work. 
He  is  an  electrical  engineer  working  on  discrete  circuit  designs  based  on  insect  vision.  His 
project  is  primarily  focused  on  models  (based  on  van  Hateren  and  Snippe)  for  luminance  and 
contrast  adaptation  at  the  early  vision  stage  (photoreceptors  and  LMCs).  He  has  also  worked 
with  Brinkworth  in  the  last  few  months  of  the  SBIR  project  on  the  construction  of  a  prototype 
optical  head  for  a  Yaw  Sensor  based  directly  on  the  models  developed  for  this  project. 

Lachlan  Dowd:  Honors  Student.  Paid  part  time  from  SBIR  funds  since  late  2004,  in  order 
to  provide  VisionEgg  programming  support  to  the  O’Carroll  laboratory. 
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